表示瓶颈
固定 PBR 图像通道能编码颜色、粗糙度和法线,却不直接暴露混合 BSDF、发光、透射、SSS、坐标依赖图案的组合逻辑。图结构让这些机制成为可编辑程序。
一个把 VLM 的结构判断、Blender 的真实执行与学习评分器的数值搜索组合起来的材质生成系统。这里不仅解释它如何工作,也逐项核对论文、代码和发布数据之间的边界。
正式标题为 ShaderAgent: Self-Evolving Agentic Procedural Material Generation。作者依次为 Ze Yuan、Chia-hao Chen、Yuqing Zhang、Yan-Pei Cao、Ding Liang、Xiaojuan Qi。第一作者署名香港大学;Xiaojuan Qi 在 PDF 首页以星号明确标记为通讯作者。
本次精读以官方 GitHub release v1.0 的 12 页 PDF为主,不填写 arXiv 编号或版本。论文第 9–10 页为参考文献,第 11–12 页为补充图,但没有独立的 Supplementary Methods。仓库自述 SIGGRAPH Asia 2026 Conference Papers,并给出 DOI 10.1145/3829340.3842164;PDF 自身仍使用 TOG 卷期与 nnnnnnn.nnnnnnn 占位 DOI。本次 Crossref 查询该仓库 DOI 返回 404,故把会议信息标为“官方仓库声明,出版元数据待核实”,不据此认定已完成正式出版。
| 资料 | 本次固定版本 | 使用边界 |
|---|---|---|
| 论文 | release v1.0 · 12 页 · 2026-09-26 下载 | 论文方法与 Table 1–3 的主要依据 |
| 官方代码 | ee84ce7b0304873024e2178a78704884e0861659 | 当前公开实现;不等同于论文实验时的私有版本 |
| MatScorer 模型卡 | 9995b00016baaaeb7ff4ad4e94ecebf2bbfef5e1 | LoRA 结构可核验;未给出完整训练超参数 |
| MatScorer 数据卡与 7 个 JSONL | 3d93f532cbfb9ba7cb2ae940cdcdc397e1b6b848 | 逐行统计列表/pair 数量;未下载 42 GiB 图像包 |
“Paper Verified”表示已核对论文原文与图表,不表示本次独立复现实验。代码审查结论另标为 Implementation Audit。
材质艺术家通常先判断光学机制,再搭节点、渲染、调参,发现机制不对时重构图。单次生成很难同时解决四件事:拓扑合理、参数合适、程序可执行、渲染接近参考。ShaderAgent 把这个制作过程组织成可以运行的反馈循环。
固定 PBR 图像通道能编码颜色、粗糙度和法线,却不直接暴露混合 BSDF、发光、透射、SSS、坐标依赖图案的组合逻辑。图结构让这些机制成为可编辑程序。
VLM 直接写 BPY 可能调用错误 socket、失效 API 或产生无效连线。DSL、静态验证与 Blender 执行诊断把错误变成可定位的反馈。
野外照片与标准材质球没有像素对齐,直接用像素/style loss 容易混入几何与光照差异。每个扰动都调用商业 VLM 又慢又不稳定,因此内环使用专门学习的 MatScorer。
核心假设:语义模型适合判断材质机制、选择参数和提出结构修改;频繁、细粒度的数值试探适合交给渲染器与低开销排序器。二者配合,应比单次生成或只有结构编辑更可靠。
Our Interpretation:这里的 self-evolving 是单个任务中候选图及生成上下文随反馈演化。没有证据表明 Designer 在运行时更新权重,也没有跨任务持久经验库、自动训练控制器或元学习算法。不能把它读成通用 recursive self-improvement。
PDF p.2PDF p.3输入可以是参考图与区域描述、文本,或用于定位的框。论文把输入统一成参考图、目标区域、细化描述与可选纹理。输出是 Blender shader graph 及其参数;不是恢复原照片的唯一真实 BRDF,也不生成物体几何。
| 符号 | 含义 | 在流程中的作用 |
|---|---|---|
| \(I_{\mathrm{ref}}\) | 原图或文本合成的参考图 | 提供外观与上下文 |
| \(B=[x_{\min},y_{\min},x_{\max},y_{\max}]\) | 目标材质区域 | Scorer 的 ROI 条件;坐标实现有待核查项 |
| \(T_{\mathrm{desc}}, I_{\mathrm{tex}}\) | 细化描述、可选无缝位图 | 描述意图;位图负责难以程序化的图案 |
| \(T_{\mathrm{dsl}},G=(V,E,\Theta)\) | DSL 文本、节点集合、连线、参数集合 | 从文本到有类型的内存图,再到 BPY |
| \(\theta\in\Theta\) | 一组材质参数取值 | 内环搜索变量;不等于 VLM 权重 |
| \(\Theta_K\subseteq\Theta\) | Inspector 选出的参数子集 | 固定预算下集中优化;代码默认最多 5 项 |
| \(\mathcal{R}(\theta)=\{R_{\mathrm{plane}},R_{\mathrm{bright}},R_{\mathrm{dark}}\}\) | 平面、亮球、暗球三种受控渲染 | 分别提供纹理、反射/表面与低光光学线索 |
| \(r_{\phi},K_{\mathrm{in}},K_{\mathrm{out}}\) | 学习的排序器、内环步数、外环轮数 | 这里以 φ 区分 scorer 权重,避免与材质 θ 混淆 |
Our Interpretation · 统一写法,非论文原始目标函数:\(\mathcal{G}_{\mathrm{valid}}\) 是能通过结构及运行时检查的图;\(U\) 表示外观偏好,而实际系统用 MatScorer 与 Critic 两套不同信号近似它。这个问题包含离散拓扑与连续参数,没有全局最优保证,也没有对 Cycles 反向传播。
本方法没有新的 latent representation 或训练过的 graph tokenizer。通用 VLM 生成文本 DSL;节点顺序是声明与引用的序列化方式,图的语义由有向连接决定。论文未提出专用自回归图概率模型或 token 级 constrained decoding;合法性主要通过提示词、解析、验证和重试保证。
PDF p.4PDF p.5| 阶段 | 模型实际读取 | 动作与产物 | 失败/反馈去向 |
|---|---|---|---|
| Enhancer / Classifier | 参考图、用户文本或目标框 | 补充描述、定位 B;选择 procedural / texture 路由 | 纯文本先合成参考图;复杂图案先合成 tileable bitmap |
| Designer | 描述、参考图、可选纹理;后续轮还有旧 DSL 和 Critic 反馈 | 生成完整 DSL,解析成 G | 语法/结构错误进入修复路径 |
| Validator + Blender | 有类型的节点图、socket、属性 | 检查拓扑,转 BPY,在 Cycles 场景执行 | 汇总错误报告给 Repairman |
| Repairman | 旧图与具体执行错误 | 移除指定节点、合并 replacement DSL、恢复合法旧边 | 目标是恢复执行正确性,不是改善外观 |
| Inspector + MatScorer | 参考图、预调渲染、DSL 与可调列表;scorer 另读 B 和三视图 | 选参数,扰动、渲染、排序,保存历史最好值 | 数值无效时跳过候选;更差候选有 5% 接受概率 |
| Critic | 预调/已调候选与参考;代码为亮球视图 | 按 rubric 比较,决定接受或提出结构建议 | 回到 Designer 生成完整新 DSL,而非局部语法补丁 |
Reasoning:视觉差异分析、材质类别与机制选择。Action:生成 DSL、局部补丁、调参数、调用渲染。Observation:验证错误、渲染图、scorer 数值与 critic rubric。Environment:Blender/Cycles 及预设场景。Trajectory:逐次参数、接受决定与渲染记录。Reward:内环兼容分,属于搜索目标而非 RL policy 的在线训练奖励。
Memory:当前任务的最好图、最好渲染和剩余问题;公开实现没有跨任务 episodic memory。Retry:DSL 本地验证重试、运行时补丁重试、整次 trial 重试是不同层级。默认三次 trial 是失败后的重试,不是无条件跑三次再择优。
按 PMG 节点流程图的表达方式整理:实线表示执行流,虚线表示共享数据。下图为独立内嵌 SVG,点击可放大查看。
放大后可在图内横向滚动;下方保留可复制的 Mermaid 源码。
实线表示执行流,虚线表示共享数据。图按已核对的代码提交 ee84ce7b0304873024e2178a78704884e0861659 整理;容器只表示 ShaderAgent 的循环范围,不表示 PMG 当前已实现这些嵌套循环。省略部分模型响应解析与调用重试细节。
%%{init: {"flowchart": {"useMaxWidth": true, "nodeSpacing": 8, "rankSpacing": 26, "diagramPadding": 6, "curve": "linear"}}}%%
flowchart TD
T[参考图 · 可选] -.图像输入.-> U[工具 · 输入准备]
P[文本描述] --> U
U -->|已有参考图| A[VLM · Classifier 路由与区域定位]
U -->|仅文本| PE[LLM · 文本增强]
PE --> IG[图像模型 · 生成参考图]
IG --> A
A -.参考图与描述及 bbox.-> REF[共享任务上下文]
A --> TR[Trial 入口 · 失败重试最多 3 次尝试]
TR --> ROUTE{材质路由}
ROUTE -->|程序化| Q
ROUTE -->|需要图案纹理| TG[图像模型 · 生成可平铺纹理]
TG --> Q
TG -.固定纹理资产.-> G
subgraph OUTER[结构改进 · 最多 3 轮]
direction TB
Q[Prompt 组装 · 任务或最好候选与反馈] --> G[VLM · Designer 生成完整 DSL]
G --> V[工具 · DSL 解析与静态校验]
V -->|不通过且有预算| DR[反馈 · 原 DSL 与校验错误]
DR -->|最多额外 2 次| G
V -->|通过| R[工具 · 转 BPY 与 Blender 预渲染]
R -->|失败且有修复预算| RP[VLM · Repairman 最小 JSON 补丁]
RP --> AP[工具 · 应用补丁与校验]
AP -->|通过 · 最多额外 2 次修复| R
R -->|成功| PRE[保存 pretune 候选]
PRE --> I[VLM · Inspector 选最多 5 个可调参数]
subgraph INNER[参数内循环 · 基线评分后最多 50 次扰动]
direction TB
INIT[初始化当前候选与历史最好] --> RV[工具 · Cycles 渲染平面与亮球及暗球]
RV --> MS[MatScorer · 参考区域与候选三视图评分]
MS --> KEEP[工具 · 接受或拒绝并维护历史最好]
KEEP --> NEXT{参数与预算检查}
NEXT -->|有参数且有剩余预算| MUT[工具 · 随机扰动所选参数]
MUT --> RV
RV -->|扰动候选渲染失败| NEXT
NEXT -->|无参数或预算耗尽| BEST[返回内循环历史最好 tuned 候选]
end
I --> INIT
BEST --> C1[VLM · Critic 比较 pretune 与 tuned]
PRE -.预调候选.-> C1
C1 --> STEP[选择本轮胜者]
STEP -->|第一轮| SAVE[保存全局最好 DSL 与渲染及反馈]
STEP -->|已有历史最好| C2[VLM · Critic 比较本轮胜者与历史最好]
C2 --> SAVE
SAVE -.历史最好候选.-> C2
SAVE --> CHECK{质量与外循环预算检查}
CHECK -->|未达标且还有轮数| BACK[循环返回 · 最好 DSL 与剩余问题]
BACK --> Q
end
REF -.任务条件.-> Q
REF -.参考信息.-> RP
REF -.参考信息.-> I
REF -.参考图与 bbox.-> MS
REF -.参考图与描述.-> C1
REF -.参考图与描述.-> C2
CHECK -->|每个评估维度均至少 0.8| OK[结束 · 交付最好 DSL 与 BPY 及渲染]
CHECK -->|轮数耗尽但有有效候选| LIMIT[结束 · 返回最好有效候选并保留未达标说明]
V -->|生成重试耗尽| TF[Trial 失败 · 保留诊断]
R -->|修复预算耗尽| TF
AP -->|补丁失败| TF
C1 -->|调用失败且内部重试无效| TF
C2 -->|调用失败且内部重试无效| TF
TF -->|还有 Trial 预算| TR
TF -->|Trial 预算耗尽| FAIL[结束 · 失败与诊断]
最小语法只有节点、属性、常量输入与连接;另支持 ColorRamp 的 ramp 和 Curve 的 point。下例是本页为讲解编写的简单灰色噪声材质,不是论文原始输出:
node Noise ShaderNodeTexNoise
in Scale=8.0
node Bump ShaderNodeBump
in Strength=0.15
node BSDF ShaderNodeBsdfPrincipled
in Base Color=(0.45, 0.45, 0.45, 1.0)
in Roughness=0.6
node Output ShaderNodeOutputMaterial
link Noise.Fac -> Bump.Height
link Bump.Normal -> BSDF.Normal
link BSDF.BSDF -> Output.Surface
in 只接收字面常量,节点间引用必须用 link。输入已被连接驱动时,其默认数值不能产生同样作用,因此 Inspector 的可调列表排除这类输入。节点类型负责执行语义;边负责计算依赖;参数控制尺度、颜色、光学响应。
静态图检查抓重复 ID、损坏端点、环和悬空节点等问题;Blender 运行时再判断实际 socket 和版本相关 API。Blender 可能自动重命名重复节点、容忍无贡献节点,所以“脚本没崩”不足以表示结构干净。反过来,静态解析也无法替代 Blender 的实际约束。
补丁由 node_ids_to_remove 与 replacement_dsl 组成。在图副本上验证成功才接受;新连线优先,再尽可能恢复仍合法的旧边。执行修复力求保留设计;外观改写允许替换机制。论文声称 DSL 平均比带 try/except 的等价 BPY 短 19.0 倍,但未在这份 PDF 中给出分布、tokenizer 或无错误处理版本的对照,不能把 19×直接等同于总 API 成本降幅。
该表示是 Blender 节点图接口,不是 SBS/SBSAR、CompactSBS 或 DiffMat。没有 Substance Designer 执行链,没有可微渲染器,也没有可训练的拓扑网络。其泛化范围仍受提示词提供的 node-signature 清单及 Blender 支持范围限制。
PDF p.5代码:graph.py:470代码:designer.py:129固定拓扑后,Inspector 从外观误差出发选少量参数。每步只提出一组新参数、渲染三视图并评分;这是顺序 stochastic hill climbing,不是每步采样大量候选的 beam search,也不是基于梯度的优化。
\(v\) 是标量参数,\(u\) 是对数空间均匀噪声。倍率约覆盖 0.1 到 10 倍,适合跨度很大的纹理尺度,但相当激进。代码对接近零的值改用加性扰动,并将标量裁剪至 \([-10^{4},10^{4}]\);这不是依据每个 Blender socket 物理范围的逐项约束。
此处省略了 scorer 的固定参考条件。新候选更好则接受,否则仍可能接受以跳出浅局部极值。相等分数也进入随机接受分支;没有随时间退火的温度。历史最好值 \(\theta^{\ast}\) 单独保存,因此当前 walker 退步不代表最终返回值退步。
颜色使用 HSV 扰动,当前配置 hue ±0.15、S/V ±0.2,alpha 不动;布尔值以 0.2 概率翻转。代码用“长度 3 或 4 且各元素都在 0–1”的启发式判断颜色,可能把某些数值向量也当 RGB;这需要复现实验时检查。默认 momentum=0,不能把函数签名中的 0.9 当成实际配置。
调参可能提高 MatScorer 却降低 Critic 偏好,所以还要比较预调与已调结果。论文写作 \(s^{\mathrm{best}}_k\gt\tau_{\mathrm{pass}}\) 即通过;当前代码则动态生成 3–6 个维度,计算加权分选择胜者,并要求每个维度都至少达到 0.8 才停止。两套阈值逻辑不能混用。
PDF p.5PDF p.6代码:tuner.py:175代码:critic.py:102MatScorer 以 Qwen3-VL-Reranker-8B 为底座,输入参考图、框及一个候选的 plane / bright_ball / dark_ball 三张渲染。每个候选独立做一次前向。模型不输出自然语言解释,直接从 yes/no 的 logit 差得到标量。
\(h_{\mathrm{last}}\) 是最后位置的隐藏表示,\(W_{\mathrm{yes}},W_{\mathrm{no}}\) 为语言模型输出头对应两词的权重。sigmoid 只把输出限制在 0–1,不保证它已校准成“人类认可概率”。训练权重 φ 与材质搜索参数 θ 是两个阶段的变量。
Our Interpretation:这是论文所述 listwise softmax cross-entropy/模型卡 InfoNCE 的标准形式,用来解释为何拉高正样本相对负样本的分数。公开材料没有训练脚本,故不能据此确认实际 loss 用 sigmoid 前后哪个张量、是否另有 temperature 或 weighting。负样本在列表中没有顺序意义。
LoRA 的公开配置是 rank=8、alpha=32、dropout=0.05,作用于语言部分 q/k/v/o 与 gate/up/down 投影。adapter_config 指向语言部分七类投影,modules_to_save 为空,bias=none;采用底座冻结、低秩增量微调的配置,未额外保存视觉编码器或输出头。此处没有 Designer task-specific fine-tuning,也没有把整个 pipeline 端到端训练。
PDF p.6代码:metrics.py:116| 项目 | Stage A:synthetic | Stage B:human-curated |
|---|---|---|
| 学习需求 | 粗类别与跨材质区分 | 同类材质的粗糙度、透射、位移等细差异 |
| 论文构造 | 约 13,800 材质库;HY3D-Bench 分部件 caption→embedding 检索→场景渲染;top-L 近邻作代理正样本,随机负样本 | primitive × 场景 × 材质组合;小扰动为正,大扰动为负;人工丢弃难以辨别/歧义排序 |
| 监督局限 | 检索先验可能错;负例常过于容易 | 人工标注规模小;审阅标准及一致性未完整公开 |
| 论文称 training lists | 57,826 | 1,007 |
| 实际公开 training rows | 11,573 个 listwise groups | 364 个 listwise groups |
| 将每个正样本与各负样本配对 | 57,826 对,精确对应论文数字 | 1,007 对,精确对应论文数字 |
| 验证/评测 | 1,318 groups → 6,586 pairs | 205 + 43 = 248 forced-choice rows |
Implementation Audit:逐行核算七个公开 JSONL 共 14,308 rows。上述数字差异主要来自 list 与 pair 的计数口径,不能据此断言训练数据缺失。合成训练中 11,534 行含 5 个负样本、39 行含 4 个;人工训练每行含 1–5 个。构造叙述是否对应同一训练版本,仍需作者说明。
rank 评测公开文件为 644 对、43 个 group,但只有 40 个不同 query 图像路径;因此“43 reference scenes”不能直接等同于 43 张独立图片。fine 对应 205+43 对;pair 不是独立场景数量。公开人工训练与两份 fine 文件在 query 路径及精确 query-positive-negatives 组合上未发现重叠;这只是文件级检查,不能排除底层材质/近重复图像或历史训练版本的重叠。
| 设置 | 已公开值 | 证据与边界 |
|---|---|---|
| 主实验 backbone | Gemini 3.1 Pro Preview | 仓库 README 说明;PDF 主体未逐角色列出 |
| 消融 backbone | gemini-3-flash-preview | PDF §5.2,降低计算成本 |
| 消融预算 | Kout=3、Kin=30 | 不能与主表的绝对分数直接比较 |
| 当前默认角色模型 | gpt-6-astra;图像生成 gpt-image-2.5-flare | 仅为下载代码的字符串配置,不等于论文设置或服务可用性验证 |
| 当前另一套配置 | claude-opus-5 + Gemini 图像服务 | 公开配置选项;本次未调用 |
| Blender / 运行环境 | Blender 4.1+、Python 3.10+、NVIDIA GPU | README;确切论文硬件未提供 |
| 主渲染 / 内环渲染 | 512×512、512 samples / 256×256、256 samples | 当前 default.yaml,Cycles GPU |
| 结构 / 调参预算 | max_steps=3、max_iter=50、max_params=5 | max_trials=3 只在 trial 失败后使用 |
| 扰动 | log_range=2.3;accept_prob=0.05;momentum=0 | 当前 YAML;候选三视图后评分 |
| 精度 | scorer bfloat16、merge_lora=true | 当前推理配置,不能推断训练精度 |
| 排序器速度 | 约 4.4 candidates/s | 论文报告;未给完整硬件与端到端测时条件 |
| 训练缺失项 | optimizer、LR、batch、steps/epochs、scheduler、warmup、weight decay、gradient accumulation、GPU 与训练时长 | 12 页 PDF 引用 supplementary;发布资产未含该文档;模型卡也未补齐 |
| 生成缺失项 | 论文确切 temperature、top-p、输出上限、随机种子控制 | 不得以服务默认值伪装成论文原始设置 |
| 评测缺失项 | held-out judge 身份与完整提示、人评人数与分配、失败样本评分分母、CI、baseline 完整适配 | 正文称详见补充材料,本次未取得 |
usage.json。本次工作只阅读与审计公开材料,没有启动 Blender、下载 8B 权重、调用付费模型或重跑主实验。HTML 的浏览器验收与论文实验复现是两件不同的事。
代码:default.yaml:1评测包含 AI 图、Blender 材质预览、Substance Designer 预览、真实照片裁剪及 CG 图,刻意提高玻璃、发光、半透明等扩展材质的比例。这个分布适合检验方法的目标能力,但并非随机抽样的普遍材质分布。
| 方法 | 可执行率 % ↑ | VLM 0–10 ↑ | 人类偏好 % ↑ | MatScorer ↑ |
|---|---|---|---|---|
| Retrieval | 100.0 | 4.80 | 16.3 | 0.128 |
| VLM-generated | 77.8 | 5.11 | 21.1 | 0.142 |
| VLMaterial | 84.4 | 2.61 | 4.1 | 0.064 |
| BlenderAlchemy | 97.8 | 4.59 | 20.7 | 0.141 |
| ShaderAgent | 100.0 | 5.33 | 37.8 | 0.180 |
人类偏好相对 VLM-generated:37.8−21.1。原文写“+17 points”为取整;不是相对提升 17%。
VLM judge 相对次优 5.11。自动 judge 的差距小于人类偏好差距,缺乏误差条时不能断言统计显著。
100% 可执行率对应本次 45 例;Retrieval 同样 100%,故“所有指标都支配”应包含并列,而非全部严格胜出。
| baseline | 比较意义 | 公平性/解释边界 |
|---|---|---|
| Retrieval | 已有材质库是否足够 | 天然可执行,但受库覆盖限制;检索系统本身不是从零生成 |
| VLM-generated | 直接 single-shot BPY 的能力 | 低可执行率支持执行约束的重要性;不等于隔离 DSL 的净增益,因为预算和调参也不同 |
| VLMaterial | 学习过的程序化图生成分布能否迁移 | 作者认为训练分布窄导致低分;未取得统一预算与适配细节,不能单归因为模型设计 |
| BlenderAlchemy | 迭代视觉编辑能否替代专用结构/参数循环 | 原任务是编辑;从零生成的初始化和重试协议未完整给出 |
| ShaderAgent | DSL+修复+调参+Critic 的组合系统 | 同时增加组件与推理计算;应进一步报告质量-成本曲线 |
五项 Human Preference 恰好总和为 100%,应读作该比较协议里的选择占比,不是两两胜率,也不是任务成功率。正文只说匿名选择可编辑结果,样本分配与分母细节缺失,不能自行补成“多名专家盲测”。MatScorer 同时是优化器和主表指标,因此这列存在指标同源性;人评与独立 VLM judge 才是额外证据,但仍需完整协议。
作者报告 BlenderAlchemy 每材质约 60–130 次 VLM 调用,ShaderAgent 约 19 次,即约 3.2–6.8 倍差异。该数字不包括所有费用、渲染耗时和显存;“VLM 调用更少”不直接等于端到端更快。按 4.4 candidates/s 估计,仅 50 次 scorer 前向约 11.4 秒,尚未加三视图渲染和外部模型。
| 配置 | VLM ↑ | MatScorer ↑ | 相对 full 的 VLM / scorer 变化 |
|---|---|---|---|
| Full | 3.79 | 0.196 | — |
| w/o Parameter Tuning | 3.40 | 0.143 | −0.39 / −0.053 |
| w/o Inspector | 3.70 | 0.157 | −0.09 / −0.039 |
| w/o Finetuned MatScorer | 3.62 | 0.161 | −0.17 / −0.035 |
| w/o Critic | 3.29 | 0.119 | −0.50 / −0.077 |
去掉 Critic 降幅最大,支持结构反馈在此设置下有用;不代表任何任务里它都最关键。去掉 Inspector 的 VLM 降幅仅 0.09,作者解释为很多图本来就没有太多旋钮。Repairman 被排除在质量消融之外,因此这张表不能证实它对成功率与成本的独立贡献。无方差、paired 检验及完整控制协议时,“独立贡献”仍需谨慎。
PDF p.7| scorer 训练 | rank PWACC ↑ | rank NDCG ↑ | fine ACC ↑ | synth loss ↓ | human loss ↓ |
|---|---|---|---|---|---|
| Pretrained | 0.6800 | 0.7908 | 0.9069 | 1.1705 | 0.5471 |
| Stage A only | 0.7485 | 0.8274 | 0.8381 | 0.2538 | 0.7368 |
| Stage B only | 0.6989 | 0.7956 | 0.9251 | 1.0234 | 0.5266 |
| Mixed, single-stage | 0.7093 | 0.7899 | 0.8378 | 0.8407 | 0.5251 |
| A → B | 0.7495 | 0.8238 | 0.9271 | 0.3021 | 0.5418 |
怎么读指标:PWACC 衡量人类偏序中的成对比较正确率;NDCG 关注排序前列与相关性增益;fine ACC 检验同材质微小差异的强制选择。两项 CE loss 衡量偏好列表目标,不能替代实际编辑质量。rank 43 group 与 fine 248 pair 也不是同一评测单位。
Paper Verified:A→B 在 PWACC 和 fine ACC 最高;比 pretrained 分别增加 6.95 和 2.02 个百分点。Stage A 单独训练把 fine ACC 从 0.9069 降至 0.8381,支持粗类别代理监督可能损害细粒度辨别。A→B 比 A-only 的 PWACC 只高 0.10 个百分点,比 B-only 的 fine ACC 只高 0.20 个百分点。
Author Claim:混合训练造成 coarse/fine 梯度“破坏性干扰”。Our Interpretation:表中显示混合策略的 rank 与 fine ACC 较弱,但没有梯度冲突测量、采样比例消融或等步数控制,尚不能唯一归因于这一机制。
以下来自固定提交与发布数据的静态检查。区分“已看到的代码行为”和“它对论文结果的影响”;没有运行完整系统时,不把代码风险说成论文结果已经失效。
| 核查项 | 论文/文档 | 公开实现与判断 |
|---|---|---|
| Critic 可见视图 | §3.4:Scorer 与 Critic 共用三视图 | pipeline._critic_views 只保留 bright_ball。发光/透射在暗球的线索未同时提供给当前 Critic。代码:pipeline.py:322 |
| 通过阈值 | 论文:best score > τpass | 代码:3–6 个 rubric 的每项 >=0.8;加权总分仅用于择优。代码:critic.py:108 |
| 结构语法错误的修复 | 论文强调局部 patch 修复 | 初次 DSL 解析失败会要求重发完整 DSL,最多初次+2 次重试;Blender 执行失败才走局部 JSON patch。代码:designer.py:36 |
| 模型与提示 | 论文主实验 Gemini 3.1 Pro Preview | 当前配置已更换模型;模板仍要求 Python/tool call,但 wrapper 末尾覆盖成 only DSL;Critic 模板 final_answer 也被末尾 only JSON 指令覆盖。代码:designer.py:43 |
| bbox 单位 | 数据卡、scorer docstring:图像像素 xyxy | Classifier 输出归一化 0–1000,tuner 原样传到 scorer,未见按图像宽高转换;需要优先核实训练预处理是否补偿。代码:tuner.py:107代码:metrics.py:183 |
| 随机性 | 配置 seed=42;代码注释称可复现 | 实际种子混入 hash(step_id),字符串 hash 默认可随进程变化;未固定 PYTHONHASHSEED 时仅固定 42 不足以跨进程复现。代码:tuner.py:97 |
| 材质先验 | 程序化生成,支持混合纹理 | 复杂图案依赖生成位图;prompt 强制优先 Displacement;节点签名由人工维护。不是无先验地搜索全部 Blender 程序。代码:prompt.py:628 |
| 数据规模 | 论文把 57,826 / 6,586 / 1,007 称 lists | 公开文件的 negative pair 总数精确匹配;应该区分 listwise groups 与 pairs。 |
| 数据构造 | 论文:top-L 邻居正样本、随机负样本 | 数据卡:实际应用的材质为正,负例按 hard/medium/easy 桶采样;不同于正文简述,需确认具体训练版本。 |
| 训练/评测分离 | 数据卡先称 eval 未训练 | 同卡另称 stage2_extra 曾混入上游 S2 训练;当前公开 train 与该 split 无精确记录/query 路径交集。历史 checkpoint 使用何版本未确认,不能据此指控或排除泄漏。 |
| 训练 recipe | README 称模型卡给完整 recipe | 模型库当前只有 README、adapter_config 和权重;完整 optimizer/训练超参及训练脚本未见。 |
| 场景与分部件应用 | 应用示例有丰富物体/场景 | PDF 明说生成 shader 后人工分配到场景对象或 mesh parts,不是自动分割、绑定和场景重建。 |
若一张 512×512 图的像素框为 [128,128,384,384],归一化表示应为 [250,250,750,750]。将后者直接按像素解释会指向不同范围。当前代码确实存在两种单位的文档/数据流冲突;实际模型是否仍正确理解,要用可视化框与固定 checkpoint 做 A/B 实验,而不能只凭注释判断。
公开性边界:代码与两套 preview 场景已公开,MatScorer adapter 与渲染数据也已发布;仓库并非“只有项目页”。但渲染图片不等于训练用原始 .blend 材质库,完整 benchmark 与训练脚本也未随本 release 提供。代码 MIT、预览场景与模型/数据各有不同条款,使用时按各资产 NOTICE 核对。
下面提供固定代码提交的原始提示词与配置,点击展开查看。它们是当前公开实现,不能当作未提供的论文实验 Supplementary prompt。运行时还会填入目标描述、图像、旧图、错误和已知 node signatures,并在部分角色末尾追加输出格式约束。
| 角色 | 输出与关键约束 | 解读 |
|---|---|---|
| Classifier | PROCEDURAL / TEXTURE;目标框 yxyx → 内部 xyxy | 控制是否引入位图生成;框必须与 scorer 坐标语义一致 |
| Designer | 完整 DSL;供给 exact node signatures | 规则约束与后验验证,不是 token grammar 解码 |
| Repairman | JSON: node_ids_to_remove + replacement_dsl | 局部图补丁;失败时对未改变的旧图重试 |
| Inspector | JSON: reasoning + params;最多 5 项 | 围绕视觉差异选数值参数;不输出最优值 |
| Critic | 3–6 项 rubric,权重和为 1 | 至少 70% 来自图像证据、至多 30% 来自描述,是 prompt 要求而非测得保证 |
| Critic feedback | 节点添加/移除/重连、enum 切换 | 避免直接指定数值,由后续 tuner 处理数值;仍是软约束 |
| MatScorer | Query 图+B,Document 三视图;yes/no head | 固定输入顺序和框编码是推理复现的重要组成 |
PROMPT_ENRICH_TEMPLATE = """You are a professional prompt enhancer specialized in material visualization and texture generation.
Your task is to rewrite the user's input into a precise image prompt that generates a clear, complete, and technically analyzable representation of a material or surface.
Core Objectives:
1. **Visual Completeness:** The target object or surface carrier must be fully contained within the frame with visible margins on all sides.
2. **Material Homogeneity:** The material must be strictly homogeneous, spatially invariant, or repetitive (tileable).
Material Constraints (Strict Enforcement):
- **Definition of Homogeneity:** Treat the material as if it will be baked into a texture map. A random crop from any part of the surface should look consistent with any other part.
- **No Unique Features:** Strictly remove or rewrite any descriptions implying a central logo, specific placement, gradient meant for a specific shape, or non-repeating decals.
- **Repetition:** If there are patterns (embroidery, scratches, grain), describe them as an "all-over pattern," "seamless tiling," or "uniform distribution."
Composition and Framing Rules:
- **Containment:** Describe the subject as "isolated" or "floating" in the center with "wide margins" or "ample negative space" around it.
- **Avoid Cropping:** Explicitly command the view to be "zoomed out slightly" or "full shot" to prevent edges from touching the image border.
- **Carrier Object:** If no object is specified, use a "flat square fabric swatch," "sphere," or "draped cloth" that is fully visible against a neutral background.
Enhancement Rules:
- Focus on physical attributes: roughness, specularity, normal map details (bumps/relief), and fiber structure.
- Lighting: Use "studio lighting" or "soft ambient occlusion" to show texture depth without harsh shadows that obscure the pattern.
Output Constraints:
- Output ONLY the final enhanced image prompt.
- Do NOT include explanations or prefixes.
User input:
{description}
"""
IMAGE_GENERATE_TEMPLATE = """You are an image generation model specialized in producing high-fidelity material reference images.
Your task is to generate an image that strictly adheres to the prompt, prioritizing material consistency and perfect framing.
CRITICAL Constraint 1: COMPOSITION & FRAMING (Zero Tolerance for Cropping)
* **Full Visibility:** The subject must be 100% visible. No part of the object, swatch, or shape usually touches the edge of the canvas.
* **Safety Margin:** Ensure there is a visible gap (padding/negative space) between the subject and the four borders of the image.
* **Center Bias:** Place the subject in the visual center, surrounded by a neutral background.
CRITICAL Constraint 2: MATERIAL HOMOGENEITY (Texture Consistency)
* **Spatial Invariance:** The material surface must be uniform. Do not generate unique variations, fading, or specific localized wear (e.g., worn edges vs. clean center). The texture must appear consistent across the entire surface.
* **Repetitive Logic:** If the prompt implies a pattern, render it as a seamless, tiling, all-over pattern.
* **No Central Focal Points:** Avoid placing a unique design or logo in the middle. The surface information should be distributed equally.
Rendering Priorities:
* **Physical Plausibility:** Accurately simulate light interaction (PBR-like qualities), surface relief, and tactile details.
* **Clarity:** Use even, neutral studio lighting. Avoid artistic vignettes or dramatic shadows that hide the material edges.
Output Requirements:
* Produce a single, coherent image.
* The subject should occupy roughly 70-80% of the canvas to ensure details are visible but edges are NOT cut off.
The prompt is:
{prompt}
"""
TEXTURE_GENERATION_TEMPLATE = """You are an expert texture artist specialized in creating seamless, high-quality PBR texture maps.
Your task is to generate a base color (albedo) texture map based on the provided reference image(s) and description.
Input Description:
{description}
Requirements:
1. **Seamlessness:** The texture must be perfectly tileable (seamless) in both X and Y directions.
2. **Flatness:** The texture should represent the surface color (albedo) without baked-in lighting, strong shadows, or specular highlights. It should be a flat scan of the material.
3. **Consistency:** The texture must match the visual characteristics (color, pattern, scale) of the material shown in the reference image(s). If multiple images are provided, compare them to distinguish true base color from lighting effects.
4. **Resolution:** High detail and sharpness.
Output:
- A single square image representing the base color texture.
"""
CLASSIFICATION_TEMPLATE = """You are a vision-language material classifier for 3D shader/texture workflows.
You are given:
- Reference image(s) showing a material on a carrier object, such as a swatch,
sphere, cloth, object part, or full object.
- An optional text description naming the part or material of interest.
Produce, in one shot:
1. One label describing how that material should be authored.
2. A tight bounding box around the target material region in the FIRST reference image.
There are only two labels, distinguished by where the visible variation lives:
- PROCEDURAL — the look is a property of the surface STRUCTURE: micro-geometry,
fiber direction, layering, anisotropy, sub-surface variation. Shader nodes
(noise / voronoi / wave / bump / displacement / anisotropy / coordinate-driven
masks) model exactly this.
- TEXTURE — the look is 2D IMAGE CONTENT painted, printed, or projected onto an
otherwise plain surface. The defining identity is the picture itself, and is
authored as a basecolor (and companion) image map.
### Input Contract
Each item is one target material. The pipeline has already filtered out
multi-material grouping and UV problems — do not reason about them. If the
description names a material, classify that one; otherwise classify the
dominant single material in the FIRST image. Text is a pointer; if it
conflicts with the image, trust the image. Multiple images show the same
material from different aspects — cross-reference them.
### STEP 1 — Classify
One question:
Is the defining variation produced by the surface STRUCTURE, or is it
IMAGE CONTENT laid on top of an otherwise plain surface?
Concrete check — the lighting test:
Imagine the material under a different lighting direction. Do the pattern's
highlights, shadows, or anisotropic streaks shift with the light?
- YES, the pattern reacts to light → STRUCTURE → PROCEDURAL.
- NO, the pattern is a flat picture and only its overall brightness changes
→ IMAGE CONTENT → TEXTURE.
Guards against common mis-classification:
- Periodicity alone decides nothing. Ribbed knit is periodic structure
(PROCEDURAL); a wallpaper motif is periodic image content (TEXTURE).
- "Natural / organic / complex-looking" is not a reason to pick TEXTURE.
Wood grain, stone veining, weathering, scales, foam, brushed metal are all
STRUCTURE — they pass the lighting test.
- Per-pixel reproduction is not required for PROCEDURAL. A semantically
faithful, editable shader is enough.
- "A photo could also represent this" is not a reason to pick TEXTURE.
- When the surface is plain except for a depicted picture or graphic, it is
TEXTURE regardless of how natural-looking the picture is.
### STEP 2 — Ground
Return an axis-aligned bounding box around the target material region in the
FIRST reference image.
- The box must enclose a region authorable by a SINGLE ShaderGraph at the
chosen label. Composite materials are fine as long as one graph (with
masks / sub-shaders) can capture them.
- Exclude background, unrelated parts, and neighboring regions that would
require a separate ShaderGraph.
- Tighten the box; do not pad.
### Output Format
Return a single JSON object, nothing else.
The bbox follows the model's native convention:
[ymin, xmin, ymax, xmax]
Use integers in [0, 1000], where:
- 0 = top or left edge
- 1000 = bottom or right edge
- ymin < ymax
- xmin < xmax
{
"reasoning": "<one short sentence: which authoring route was chosen and why, and what region the bbox encloses>",
"bbox": [ymin, xmin, ymax, xmax],
"label": "PROCEDURAL" | "TEXTURE"
}
No markdown fences, no prose outside the JSON.
"""
BLENDER_SHADER_NODE_SIGNATURES = """ShaderNodeTexCoord
out Generated (VECTOR)
out Normal (VECTOR)
out Object (VECTOR)
out Camera (VECTOR)
out Window (VECTOR)
out Reflection (VECTOR)
# NOTE: UV and Generated output is NOT available (meshes lack proper UV unwrapping, and may have different scale).
# Use ONLY the Object output for spatial coordinates.
ShaderNodeBsdfPrincipled
prop distribution='GGX'
in Base Color (RGBA)
in Metallic (VALUE)
in Roughness (VALUE)
in IOR (VALUE)
in Alpha (VALUE)
in Normal (VECTOR)
in Subsurface Weight (VALUE)
in Subsurface Radius (VECTOR)
in Subsurface Scale (VALUE)
in Subsurface Anisotropy (VALUE)
in Specular IOR Level (VALUE)
in Specular Tint (RGBA)
in Anisotropic (VALUE)
in Anisotropic Rotation (VALUE)
in Tangent (VECTOR)
in Transmission Weight (VALUE)
in Coat Weight (VALUE)
in Coat Roughness (VALUE)
in Coat IOR (VALUE)
in Coat Tint (RGBA)
in Coat Normal (VECTOR)
in Sheen Weight (VALUE)
in Sheen Roughness (VALUE)
in Sheen Tint (RGBA)
in Emission Color (RGBA)
in Emission Strength (VALUE)
out BSDF (SHADER)
# Dynamic Rules:
# * IF subsurface_method=='RANDOM_WALK_SKIN': ADD Input 'Subsurface IOR'
# * IF subsurface_method=='BURLEY': REMOVE Input 'Subsurface Anisotropy'
ShaderNodeDisplacement
prop space='OBJECT'
in Height (VALUE)
in Midlevel (VALUE)
in Scale (VALUE)
in Normal (VECTOR)
out Displacement (VECTOR)
ShaderNodeOutputMaterial
prop target='ALL'
in Surface (SHADER)
in Volume (SHADER)
in Displacement (VECTOR)
ShaderNodeCombineXYZ
in X (VALUE)
in Y (VALUE)
in Z (VALUE)
out Vector (VECTOR)
ShaderNodeMath
in Value (VALUE)
in Value (VALUE)
out Value (VALUE)
# Dynamic Rules:
# * IF operation=='RADIANS': REMOVE Input 'Value'
# * IF operation=='MULTIPLY_ADD': ADD Input 'Value'
ShaderNodeVectorMath
in Vector (VECTOR)
in Vector (VECTOR)
out Vector (VECTOR)
# Dynamic Rules:
# * IF operation=='NORMALIZE': REMOVE Input 'Vector'
# * IF operation=='DOT_PRODUCT': ADD Output 'Value', REMOVE Output 'Vector'
# * IF operation=='SCALE': ADD Input 'Scale', REMOVE Input 'Vector'
ShaderNodeVectorRotate
in Vector (VECTOR)
in Center (VECTOR)
in Axis (VECTOR)
in Angle (VALUE)
out Vector (VECTOR)
# Dynamic Rules:
# * IF rotation_type=='EULER_XYZ': ADD Input 'Rotation', REMOVE Input 'Axis', REMOVE Input 'Angle'
ShaderNodeTexImage
prop projection='FLAT'
prop interpolation='Linear'
prop extension='REPEAT'
in Vector (VECTOR)
out Color (RGBA)
out Alpha (VALUE)
ShaderNodeMix
prop data_type='RGBA'
prop factor_mode='UNIFORM'
prop blend_type='MULTIPLY'
in Factor (VALUE)
in A (VALUE)
in B (VALUE)
out Result (VALUE)
ShaderNodeBrightContrast
in Color (RGBA)
in Bright (VALUE)
in Contrast (VALUE)
out Color (RGBA)
ShaderNodeHueSaturation
in Hue (VALUE)
in Saturation (VALUE)
in Value (VALUE)
in Fac (VALUE)
in Color (RGBA)
out Color (RGBA)
ShaderNodeInvert
in Fac (VALUE)
in Color (RGBA)
out Color (RGBA)
ShaderNodeNormalMap
prop space='TANGENT'
in Strength (VALUE)
in Color (RGBA)
out Normal (VECTOR)
ShaderNodeRGB
out Color (RGBA)
ShaderNodeBump
in Strength (VALUE)
in Distance (VALUE)
in Height (VALUE)
in Normal (VECTOR)
out Normal (VECTOR)
ShaderNodeLayerWeight
in Blend (VALUE)
in Normal (VECTOR)
out Fresnel (VALUE)
out Facing (VALUE)
ShaderNodeTexVoronoi
in Vector (VECTOR)
in Scale (VALUE)
in Detail (VALUE)
in Roughness (VALUE)
in Lacunarity (VALUE)
in Randomness (VALUE)
out Distance (VALUE)
out Color (RGBA)
out Position (VECTOR)
# Dynamic Rules:
# * IF voronoi_dimensions=='4D' AND distance=='EUCLIDEAN' AND feature=='F1': ADD Input 'W', ADD Output 'W'
# * IF voronoi_dimensions=='4D' AND distance=='EUCLIDEAN' AND feature=='SMOOTH_F1': ADD Input 'W', ADD Input 'Smoothness', ADD Output 'W'
# * IF voronoi_dimensions=='3D' AND distance=='EUCLIDEAN' AND feature=='SMOOTH_F1': ADD Input 'Smoothness'
# * IF voronoi_dimensions=='3D' AND distance=='EUCLIDEAN' AND feature=='DISTANCE_TO_EDGE': REMOVE Output 'Color', REMOVE Output 'Position'
# * IF voronoi_dimensions=='3D' AND distance=='MINKOWSKI' AND feature=='F1': ADD Input 'Exponent'
ShaderNodeMapping
prop vector_type='POINT'
in Vector (VECTOR)
in Location (VECTOR)
in Rotation (VECTOR)
in Scale (VECTOR)
out Vector (VECTOR)
ShaderNodeMixShader
in Fac (VALUE)
in Shader (SHADER)
in Shader (SHADER)
out Shader (SHADER)
ShaderNodeValToRGB
in Fac (VALUE)
out Color (RGBA)
out Alpha (VALUE)
ShaderNodeAddShader
in Shader (SHADER)
in Shader (SHADER)
out Shader (SHADER)
ShaderNodeFresnel
in IOR (VALUE)
in Normal (VECTOR)
out Fac (VALUE)
ShaderNodeTexNoise
in Vector (VECTOR)
in Scale (VALUE)
in Detail (VALUE)
in Roughness (VALUE)
in Lacunarity (VALUE)
in Distortion (VALUE)
out Fac (VALUE)
out Color (RGBA)
# Dynamic Rules:
# * IF noise_dimensions=='4D' AND noise_type=='FBM': ADD Input 'W'
# * IF noise_dimensions=='3D' AND noise_type=='HYBRID_MULTIFRACTAL': ADD Input 'Offset', ADD Input 'Gain'
ShaderNodeNormal
in Normal (VECTOR)
out Normal (VECTOR)
out Dot (VALUE)
ShaderNodeNewGeometry
out Position (VECTOR)
out Normal (VECTOR)
out Tangent (VECTOR)
out True Normal (VECTOR)
out Incoming (VECTOR)
out Parametric (VECTOR)
out Backfacing (VALUE)
out Pointiness (VALUE)
out Random Per Island (VALUE)
ShaderNodeUVMap
out UV (VECTOR)
# WARNING: Do NOT use this node. Meshes lack proper UV unwrapping and may have different scale.
# Use ShaderNodeTexCoord with the Object output instead.
ShaderNodeSeparateXYZ
in Vector (VECTOR)
out X (VALUE)
out Y (VALUE)
out Z (VALUE)
ShaderNodeSeparateColor
prop mode='RGB'
in Color (RGBA)
out Red (VALUE)
out Green (VALUE)
out Blue (VALUE)
ShaderNodeCombineColor
prop mode='RGB'
in Red (VALUE)
in Green (VALUE)
in Blue (VALUE)
out Color (RGBA)
ShaderNodeObjectInfo
out Location (VECTOR)
out Color (RGBA)
out Alpha (VALUE)
out Object Index (VALUE)
out Material Index (VALUE)
out Random (VALUE)
ShaderNodeVectorTransform
prop vector_type='NORMAL'
prop convert_from='OBJECT'
prop convert_to='WORLD'
in Vector (VECTOR)
out Vector (VECTOR)
ShaderNodeValue
out Value (VALUE)
ShaderNodeTexChecker
in Vector (VECTOR)
in Color1 (RGBA)
in Color2 (RGBA)
in Scale (VALUE)
out Color (RGBA)
out Fac (VALUE)
ShaderNodeClamp
prop clamp_type='MINMAX'
in Value (VALUE)
in Min (VALUE)
in Max (VALUE)
out Result (VALUE)
ShaderNodeEmission
in Color (RGBA)
in Strength (VALUE)
out Emission (SHADER)
ShaderNodeRGBCurve
in Fac (VALUE)
in Color (RGBA)
out Color (RGBA)
ShaderNodeTexWave
prop wave_type='BANDS'
prop bands_direction='X'
prop rings_direction='X'
prop wave_profile='SIN'
in Vector (VECTOR)
in Scale (VALUE)
in Distortion (VALUE)
in Detail (VALUE)
in Detail Scale (VALUE)
in Detail Roughness (VALUE)
in Phase Offset (VALUE)
out Color (RGBA)
out Fac (VALUE)
ShaderNodeMapRange
prop interpolation_type='LINEAR'
prop data_type='FLOAT'
in Value (VALUE)
in From Min (VALUE)
in From Max (VALUE)
in To Min (VALUE)
in To Max (VALUE)
out Result (VALUE)
ShaderNodeVolumeAbsorption
in Color (RGBA)
in Density (VALUE)
out Volume (SHADER)
ShaderNodeTexGradient
prop gradient_type='LINEAR'
in Vector (VECTOR)
out Color (RGBA)
out Fac (VALUE)
ShaderNodeTexWhiteNoise
prop noise_dimensions='3D'
in Vector (VECTOR)
out Value (VALUE)
out Color (RGBA)
ShaderNodeBevel
in Radius (VALUE)
in Normal (VECTOR)
out Normal (VECTOR)
ShaderNodeTexMagic
in Vector (VECTOR)
in Scale (VALUE)
in Distortion (VALUE)
out Color (RGBA)
out Fac (VALUE)
ShaderNodeTexBrick
in Vector (VECTOR)
in Color1 (RGBA)
in Color2 (RGBA)
in Mortar (RGBA)
in Scale (VALUE)
in Mortar Size (VALUE)
in Mortar Smooth (VALUE)
in Bias (VALUE)
in Brick Width (VALUE)
in Row Height (VALUE)
out Color (RGBA)
out Fac (VALUE)
ShaderNodeFloatCurve
in Factor (VALUE)
in Value (VALUE)
out Value (VALUE)
ShaderNodeVectorCurve
in Fac (VALUE)
in Vector (VECTOR)
out Vector (VECTOR)
ShaderNodeAmbientOcclusion
in Color (RGBA)
in Distance (VALUE)
in Normal (VECTOR)
out Color (RGBA)
out AO (VALUE)
ShaderNodeCombineHSV
in H (VALUE)
in S (VALUE)
in V (VALUE)
out Color (RGBA)
ShaderNodeSeparateHSV
in Color (RGBA)
out H (VALUE)
out S (VALUE)
out V (VALUE)
ShaderNodeVectorDisplacement
prop space='TANGENT'
in Vector (RGBA)
in Midlevel (VALUE)
in Scale (VALUE)
out Displacement (VECTOR)
ShaderNodeGroup
prop name=<name>
out BSDF (Shader)
out Displacement (Vector)
out Volume (Shader)
# placeholder for subgraph node groups"""
COMMON_DSL_BLOCK = f"""----------------------------------------------------------------
### DSL Syntax
1. Define a Node:
node <node_id> <NodeType>
prop <property_name>=<value>
in <input_socket_name>=<literal_default_value>
2. Define a Link (node-to-node connection):
link <source_node_id>.<output_socket> -> <target_node_id>.<input_socket>
3. Special Node Commands:
- For ShaderNodeValToRGB (Color Ramp):
ramp <color> <position>
Example: ramp '#FFFFFF' 0.5
* Note: The node usage needs to clear default elements first. The `ramp` commands will add elements.
- For ShaderNodeRGBCurve, ShaderNodeVectorCurve, ShaderNodeFloatCurve:
point <channel_C_R_G_B_or_X_Y_Z> <x> <y>
Example: point 'C' 0.0 0.2
* Note: 'C' is the combined channel for RGB curves.
**CRITICAL RULES:**
- `in` values MUST be literal constants (numbers, tuples, color hex strings).
NEVER write node references like `SomeNode.Output` in an `in` field.
- All node-to-node connections MUST use `link` statements.
- If an input receives its value from another node, do NOT set a default for it
in the `in` line; just write the `link` instead.
Correct Example:
```
node Noise ShaderNodeTexNoise
in Scale=50.0
node Disp ShaderNodeDisplacement
in Scale=0.02
in Midlevel=0.5
node BSDF ShaderNodeBsdfPrincipled
in Base Color=(0.8, 0.2, 0.1, 1.0)
in Roughness=0.5
node Output ShaderNodeOutputMaterial
link Noise.Fac -> Disp.Height
link BSDF.BSDF -> Output.Surface
link Disp.Displacement -> Output.Displacement
```
----------------------------------------------------------------
### Available Nodes & Signatures
Use ONLY the nodes, properties, and sockets defined below.
You MUST NOT invent nodes, sockets, or properties that are not listed.
{BLENDER_SHADER_NODE_SIGNATURES}
----------------------------------------------------------------
### Surface Micro-Structure Guideline
For surface micro-details (cracks, pores, grain, relief, bumps, engravings,
ridges, scratches, etc.):
- Always prefer ShaderNodeDisplacement connected to
ShaderNodeOutputMaterial.Displacement to produce real geometry.
Performance is NOT a concern (we always render with Cycles).
- Use ShaderNodeBump / ShaderNodeNormalMap ONLY for extremely subtle,
purely visual detail that does not warrant geometric displacement.
- If an existing graph uses Bump/NormalMap for noticeable surface structure,
prefer replacing it with Displacement.
"""
SHADER_REFINE_FULL_TEMPLATE = """You are an expert technical artist refining a Blender shader graph.
You will be given:
- The target material description.
- Reference image(s) — physical evidence of the target.
- The previous DSL (the current best graph for this task).
- Critic feedback describing the remaining visual mismatches.
Your job: emit a COMPLETE new DSL that closes those gaps. Keep, modify, or
replace any part of the previous structure — change as little or as much as
the visual gap warrants.
{role_context}
""" + COMMON_DSL_BLOCK + """
----------------------------------------------------------------
### Previous DSL
```
{previous_dsl}
```
----------------------------------------------------------------
### Critic Feedback
```
{feedback}
```
----------------------------------------------------------------
### Task
{description}
----------------------------------------------------------------
### Output
Write Python code that:
1. Builds a string holding the FULL new DSL.
2. Calls `ShaderGraph_DSL_Parser(dsl_string=...)`.
3. Returns the resulting ShaderGraph as the final answer.
Do NOT `import` any module. Plain string + tool call is all you need.
"""
SHADER_GENERATION_TEMPLATE = """You are an expert technical artist specializing in Blender shader nodes.
Your task is to generate a shader graph definition for the target material,
faithfully reproducing its material behavior as observed in the provided reference image(s).
You must use the following Domain Specific Language (DSL) to define the graph.
""" + COMMON_DSL_BLOCK + """
----------------------------------------------------------------
### Task Context
Task:
{description}
Reference Image(s):
Use all provided reference images to infer the physical and visual properties
of the target material. Treat the images as physical evidence, not as style
inspiration.
----------------------------------------------------------------
### Required Physical Reasoning Order
You MUST internally reason about the material using the following fixed order.
You may decide that certain steps are NOT APPLICABLE for this material, but
you must not violate the order.
0. Spatial-variation analysis:
- For the visible surface, decide whether each appearance aspect (color,
roughness, normal/relief, metallic, etc.) is:
(a) spatially uniform,
(b) driven by procedural noise / patterns (no geometry correlation), or
(c) correlated with the mesh's own geometry — i.e. with curvature,
surface normal direction, ambient occlusion / cavity depth,
view-angle (Fresnel), object-space height/axis, or distance to
nearest edge.
- This decision determines, for each subsequent step, what feeds the
corresponding BSDF input: a constant, a procedural texture, or a
geometry-derived signal (Pointiness / AmbientOcclusion / Fresnel /
LayerWeight / Bevel / object coordinate). Plan the masks/inputs
accordingly before filling channel values.
1. Metalness classification:
- Determine whether the material is metallic (0 or 1).
- This decision constrains all subsequent reasoning.
2. Surface structure:
- Follow the "Surface Micro-Structure Guideline" above: always use
ShaderNodeDisplacement for surface micro-details (cracks, pores, grain,
relief). Bump/NormalMap only for negligible visual-only detail.
3. Specular response:
- Roughness
- IOR
- Anisotropy
- Clear coat / sheen (if applicable)
4. Base appearance:
- Base Color (diffuse for non-metals, specular color for metals)
- Emission (if present)
5. Subsurface effects:
- Subsurface scattering (only if physically plausible)
6. Internal / volumetric effects:
- Transmission
- Volume absorption or scattering (only if physically plausible)
You MUST NOT introduce effects that contradict earlier decisions
(e.g. subsurface effects on a clearly metallic surface).
----------------------------------------------------------------
### Graph Design Constraints
- Every node in the graph MUST contribute to the final output.
- Do NOT include unused or decorative nodes.
- Avoid relying on default input values to hide missing logic.
- The graph structure should directly correspond to observable visual features
in the reference image(s).
----------------------------------------------------------------
### Output Instructions
1. Briefly explain your understanding of the material
and which physical properties you aim to reproduce.
2. Describe the planned node structure at a high level
(key nodes and their roles).
3. Generate Python code that:
a. Constructs a string variable containing ONLY valid DSL code.
b. Calls the tool `ShaderGraph_DSL_Parser(dsl_string=...)` with that string.
c. Ends with the tool call so that the ShaderGraph object is returned.
IMPORTANT:
- The DSL must strictly follow the syntax and available signatures.
- The final output MUST result in a valid ShaderGraph object.
- Do NOT `import` any module. Build the DSL as a plain string and call the
tool — that's all the runtime needs.
"""
DERIVE_PBR_TEMPLATE = """You are an expert technical artist specializing in PBR material creation and Blender shader nodes.
Your task is to generate a shader graph that derives a full PBR material (Roughness, Metallic, etc.) from a provided Base Color texture map.
You will be provided with:
1. Reference image(s) of the material.
2. A user prompt that points at the wanted material on a specific part.
3. A generated Base Color texture map.
Your Goal:
Create a shader graph using the provided DSL that:
1. Starts with an `ShaderNodeTexImage` node representing the Base Color.
* **CRITICAL:** For the `image` in this Image Texture node, just use a python-style comment placeholder `# Placeholder for Base Color texture` instead of actual image loading code. I will replace it later.
2. Derives other PBR channels (Roughness, Metallic, Normal, Bump, etc.) based on the visual appearance in the reference image(s) and the semantic properties of the material.
* You can use constant values if the property is uniform.
* Based on the semantic information, you can also derive values from the Base Color using math nodes (e.g., `ShaderNodeMath`, `ShaderNodeValToRGB`, `ShaderNodeSeparateColor`, `ShaderNodeInvert`) if the surface details align with the color pattern (e.g., embroidery may bump out of the background cloth).
3. There should be a `ShaderNodeOutputMaterial` in the end.
""" + COMMON_DSL_BLOCK + """
### Task
**User Description:** {description}
### Instructions
1. **Analyze the Material:** Look at the reference image(s). If multiple images are provided, they may describe the same material from different aspects (e.g., different lighting directions, angles, or close-ups). Contrast these views to distinguish between temporary lighting effects (like specular highlights) and fixed material properties (like albedo and roughness). How does the material react to light? Is it metallic? Is it rough or shiny? Does the surface relief (bump/normal) match the color pattern? Any other extended attributes, e.g. Subsurface Scattering, emission, transparency?
2. **Design the Graph:**
* Create the `ShaderNodeTexImage` for the Base Color (remember the placeholder).
* Decide how to generate Roughness, Metallic, Normal, and e.t.c.
* Example: If it's a gold thread embroidery on cloth:
* Separate the gold color from the background (maybe using `ShaderNodeSeparateColor` or `ShaderNodeValToRGB`).
* Use this mask to drive Metallic (1 for gold, 0 for cloth).
* Use this mask to drive Roughness (shiny gold, rough cloth).
* Use this mask to drive Bump/Normal (embroidery is raised).
3. **Generate Code:** Write a Python script that:
a. Constructs a string variable containing the DSL code.
b. Calls the tool `ShaderGraph_DSL_Parser(dsl_string=...)` with that string.
c. The tool will validate the DSL and return the `ShaderGraph` object.
**Important:**
- Ensure your DSL string is valid.
- The Python code must end with the tool call.
- Do NOT `import` any module — build the DSL string and call the tool.
"""
CRITIC_PAIRWISE_PROMPT = """You are an expert Technical Artist and Material Supervisor.
Pick which of two procedurally generated material candidates more faithfully
reproduces the reference image(s).
References: {ref_indices}.
Candidate A ({a_label}): {a_view_indices}
Candidate B ({b_label}): {b_view_indices}
User Description: {description}
### Comparison rules
- Cross-reference all reference images to fix the material's intrinsic ground truth.
- Each candidate is shown as a single bright_ball render (color/specular/normal under direct light).
- Decouple lighting: ignore differences in shadow direction, reflected colors, or illumination intensity. Judge only intrinsic properties (Albedo, Surface Structure, Roughness, Metallic, Sheen, …).
- The reference list may include a base-color texture map alongside photographs; treat every reference image as ground truth.
### Evaluation protocol
Phase 1 — Decompose the description into 3–6 scoring points. Assign each a positive `weight`; weights MUST sum to 1.0.
Weight sourcing: ≥ 70 %% from what the Reference Image(s) actually show, ≤ 30 %% from the User Description (description picks *which* traits matter; never let its adjectives override the image).
> Example — "Weathered red clay brick with light mortar, surface pitting":
> | # | Question | max |
> |---|-----------------------------------------------------------|-----|
> | 1 | Muted red/clay base color matching the reference? | 0.30|
> | 2 | Light mortar at correct scale/width? | 0.25|
> | 3 | Visible surface pores/pitting? | 0.25|
> | 4 | Subtle roughness variation across the surface? | 0.20|
Phase 2 — Score A and B independently on the SAME rubric. For each point, assign normalized scores in [0, 1]:
- Full marks: trait clearly present, matches the reference.
- Partial: trait present but noticeably off (wrong scale, intensity, hue).
- Zero: trait missing or contradicting the reference.
Phase 3 — Decide:
- `scores.A` = sum of A's per-point scores (in [0, 1]).
- `scores.B` = sum of B's per-point scores (in [0, 1]).
- `winner` = "A" or "B" from the weighted totals; ties must choose "A".
- `good_enough` = true iff the winner scores >= {good_enough_min} x max on EVERY scoring point.
- `winner_remaining_issues` = structural advice (see Feedback constraint) for whatever the winner still gets less than full marks on. Empty string if `good_enough`.
### Feedback constraint
`winner_remaining_issues` drives a downstream rewrite agent that controls graph STRUCTURE only.
- Speak in terms of: nodes to add / remove / rewire, enum switches (`wave_type`,
`bands_direction`, `blend_type`, `noise_type`, `feature` …), where in the
graph the missing channel should plug in (e.g. "the rib pattern needs to
drive the Displacement input, not just Base Color").
- Do NOT prescribe numeric magnitudes — a parameter tuner explores those.
"Add a high-frequency noise channel feeding Bump.Height" is good;
"set Noise.Scale=500" is not.
### Output
Return one `final_answer(...)` call:
final_answer({{
"winner": "A" | "B",
"rubric": [{{"dimension": "...", "weight": <float>, "score_a": <float in [0,1]>, "score_b": <float in [0,1]>, "evidence": "..."}}],
"scores": {{"A": <weighted total in [0,1]>, "B": <weighted total in [0,1]}},
"reason": "<one-sentence why winner beat the other>",
"good_enough": <bool>,
"winner_remaining_issues": "<structural advice, or '' if good_enough>"
}})
"""
INSPECTOR_TEMPLATE = """You are a shader technical artist assistant.
Pick the numeric parameters most worth tuning to close the gap between the
current render and the target material.
You are given (in order):
Image 1: the reference image showing the target material.
Image 2: the current rendered output of the graph below.
Compare Image 2 to the target material in Image 1 — that comparison drives
the selection. The textual description below disambiguates which material in
Image 1 to focus on (the reference may show multiple regions).
### Target Material Description
{description}
### Current Shader Graph (DSL)
```
{graph_dsl}
```
### Tunable Parameters
`node_id.param_name = current_value`. Link-driven inputs are already excluded.
{param_list}
### Instructions
1. Identify the largest visual gaps between the target material in the
reference and the current render (color, displacement depth, pattern
scale, roughness, anisotropy, …).
2. Map each gap to specific parameters in the tunable list. Prefer params
that directly control the mismatched dimension (e.g. pattern-scale gap →
a `Scale` on a procedural texture; displacement-depth gap → a `Disp.Scale`
or `Bump.Strength`).
3. Pick at most {max_params}.
Return ONLY this JSON object:
{{
"reasoning": "<≤2 sentences: main visual gaps and why these params target them>",
"params": [{{"node": "noise1", "param": "Scale"}}, ...]
}}
"""
import json
from typing import Optional, Sequence
from PIL import Image
from ..config import AgentConfig
from ..io import get_logger
from ..llm.runtime import create_client
from ..usage import get_active
from ..render.blender import LoopException
from ..prompt import (
DERIVE_PBR_TEMPLATE,
SHADER_GENERATION_TEMPLATE,
SHADER_REFINE_FULL_TEMPLATE,
COMMON_DSL_BLOCK,
)
from ..dsl.graph import ReplaceShaderNodesTool, ShaderGraph
logger = get_logger(__name__)
class SubMaterialAgent:
def __init__(self, model_config: AgentConfig):
self.model = create_client(model_config)
# Diagnostics for the caller. ``last_raw_dsl`` is kept even when the
# DSL failed validation, since it is discarded everywhere else.
self.last_dsl_attempts = 0
self.last_raw_dsl: Optional[str] = None
DSL_RETRY_BUDGET: int = 2
def _generate_graph(self, prompt: str, images: Sequence[Image.Image]) -> ShaderGraph:
return self._generate_graph_lightweight(prompt, images)
def _generate_graph_lightweight(
self, prompt: str, images: Sequence[Image.Image],
) -> ShaderGraph:
"""Request a DSL, validating locally and re-prompting on failure.
Leaves ``last_dsl_attempts`` / ``last_raw_dsl`` set for the caller to
log, including on the raise.
"""
base = (
prompt
+ "\n\nOutput ONLY the complete DSL. Do not use Python, tools, markdown fences, or commentary."
)
attempt_prompt = base
last_errors: list[str] = []
self.last_dsl_attempts = 0
for attempt in range(self.DSL_RETRY_BUDGET + 1):
raw_dsl = self.model.complete(attempt_prompt, images)
self.last_dsl_attempts = attempt + 1
self.last_raw_dsl = raw_dsl
if attempt:
collector = get_active()
if collector is not None:
collector.bump("dsl_validation_retries")
graph = ShaderGraph.from_dsl(raw_dsl)
if not graph.errors:
graph._raw_dsl = raw_dsl
return graph
last_errors = list(graph.errors)
if attempt == self.DSL_RETRY_BUDGET:
break
logger.warning(
"DSL validation failed (attempt %d/%d): %s — re-prompting.",
attempt + 1, self.DSL_RETRY_BUDGET + 1, "; ".join(last_errors[:3]),
)
attempt_prompt = (
base
+ "\n\nYour previous DSL was rejected by the local validator.\n"
+ "Previous DSL:\n" + raw_dsl
+ "\n\nValidator errors:\n- " + "\n- ".join(last_errors)
+ "\n\nEmit a corrected, complete DSL. Every node must reach an "
"output node; remove or wire up any dangling node."
)
raise LoopException(
f"DSL has structural errors after {self.DSL_RETRY_BUDGET + 1} attempts:\n- "
+ "\n- ".join(last_errors)
)
def generate_procedural_graph(
self,
description: str,
reference_images: Sequence[Image.Image],
error_feedback: Optional[str] = None,
critic_feedback: Optional[str] = None,
previous_graph: Optional[ShaderGraph] = None,
) -> ShaderGraph:
# Only the in-step repair sub-loop in pipeline.py passes error_feedback.
if error_feedback and previous_graph is not None:
return self._repair_graph(
description, reference_images, previous_graph,
error_feedback=error_feedback,
)
if critic_feedback and previous_graph is not None:
return self._refine_graph_full(
description, reference_images, previous_graph,
critic_feedback=critic_feedback,
)
prompt = self._build_procedural_prompt(description)
logger.info(f"Shader graph generating...\n\nDescription: {description}")
res = self._generate_graph(prompt, reference_images)
return res
def _refine_graph_full(
self,
description: str,
reference_images: Sequence[Image.Image],
previous_graph: ShaderGraph,
critic_feedback: str,
role_context: str = "",
) -> ShaderGraph:
"""Whole-DSL rewrite seeded by the previous graph's DSL."""
prompt = SHADER_REFINE_FULL_TEMPLATE.format(
description=description,
previous_dsl=previous_graph.to_dsl(),
feedback=critic_feedback,
role_context=role_context,
)
logger.info(
"Shader graph refining (full rewrite)...\nDescription: %s\nFeedback: %s",
description[:80], critic_feedback[:200],
)
res = self._generate_graph(prompt, reference_images)
return res
def _repair_graph(
self,
description: str,
reference_images: Sequence[Image.Image],
previous_graph: ShaderGraph,
error_feedback: str,
role_context: str = "",
) -> ShaderGraph:
"""Request a minimal JSON patch and validate it transactionally."""
schema = {
"type": "object",
"properties": {
"node_ids_to_remove": {
"type": "array", "items": {"type": "string", "minLength": 1},
"uniqueItems": True,
},
"replacement_dsl": {"type": "string"},
},
"required": ["node_ids_to_remove", "replacement_dsl"],
"additionalProperties": False,
}
prompt = (
"Repair only the broken shader nodes with a minimal local patch. "
"Keep working nodes intact. Return ONLY a JSON object matching the schema.\n"
+ role_context + "\n" + COMMON_DSL_BLOCK
+ "\nTask:\n" + description
+ "\nPrevious graph (DSL):\n" + previous_graph.to_dsl()
+ "\nBlender/validator error:\n" + error_feedback
+ "\nReplacement DSL contains only fixed/new nodes and links. Links may "
"reference surviving nodes from the previous graph. To redefine an ID, "
"include it in node_ids_to_remove. Removal IDs must exist and be unique. "
"Valid old links are restored when both endpoints survive and the target "
"has capacity; explicit replacement links take priority. "
"Return JSON data, with DSL in replacement_dsl, rather than Python or a complete rewrite.\n"
+ "JSON schema:\n" + json.dumps(schema)
)
logger.info(
"Shader graph repairing...\nDescription: %s\nError: %s",
description[:80], error_feedback[:200],
)
attempt_prompt = prompt
last_error = ""
self.last_dsl_attempts = 0
for attempt in range(self.DSL_RETRY_BUDGET + 1):
raw_patch = self.model.complete(attempt_prompt, reference_images)
self.last_dsl_attempts = attempt + 1
self.last_raw_dsl = raw_patch
if attempt:
collector = get_active()
if collector is not None:
collector.bump("dsl_validation_retries")
try:
payload = json.loads(raw_patch)
if not isinstance(payload, dict) or set(payload) != set(schema["required"]):
raise ValueError("Patch must contain exactly node_ids_to_remove and replacement_dsl.")
return ReplaceShaderNodesTool().forward(previous_graph, **payload)
except (ValueError, TypeError) as error:
last_error = str(error)
attempt_prompt = (
prompt + "\nPrevious JSON patch rejected by the local validator:\n"
+ raw_patch + "\nValidator errors:\n" + last_error
+ "\nReturn a corrected JSON patch against the unchanged previous graph."
)
raise LoopException(
f"Graph repair failed after {self.DSL_RETRY_BUDGET + 1} attempts:\n{last_error}"
)
def _build_procedural_prompt(self, description: str) -> str:
return SHADER_GENERATION_TEMPLATE.format(description=description)
_PBR_ROLE_CONTEXT = (
"You are working on a PBR texture-derivation graph that derives "
"Roughness, Metallic, Normal, etc. from a Base Color texture. "
"The ShaderNodeTexImage for Base Color is correct; focus on "
"the PBR derivation nodes."
)
def derive_pbr_graph(
self,
description: str,
reference_images: Sequence[Image.Image],
texture_map: Image.Image,
error_feedback: Optional[str] = None,
critic_feedback: Optional[str] = None,
previous_graph: Optional[ShaderGraph] = None,
) -> ShaderGraph:
if error_feedback and previous_graph is not None:
return self._repair_graph(
description=description,
reference_images=reference_images,
previous_graph=previous_graph,
error_feedback=error_feedback,
role_context=self._PBR_ROLE_CONTEXT,
)
if critic_feedback and previous_graph is not None:
return self._refine_graph_full(
description=description,
reference_images=reference_images,
previous_graph=previous_graph,
critic_feedback=critic_feedback,
role_context=self._PBR_ROLE_CONTEXT,
)
prompt = self._build_texture_prompt(description)
logger.info("PBR attributes aligning...")
res = self._generate_graph(prompt, list(reference_images) + [texture_map])
return res
def _build_texture_prompt(
self,
description: str,
) -> str:
return DERIVE_PBR_TEMPLATE.format(
description=description,
)
# OpenAI-compatible Chat Completions. export OPENAI_API_KEY=...
# Chat: gpt-6-astra. Texture maps: gpt-image-2.5-flare (same key, Images API).
pipeline:
max_trials: 3 # whole-pipeline retries after a failed trial
max_steps: 3 # Designer rewrite / structure-refinement rounds per trial
param_tuning_enabled: true
dataset_workers: 1
dataset_indices_path: data/index.jsonl
blender:
executable_path: blender
render:
engine: CYCLES
samples: 512
resolution: [512, 512]
device: GPU
compute_device_type: CUDA # or OPTIX
camera:
azimuth_angles: [45]
elevation: 20
scene:
# Hero view for the final PNG. Critic/tuner also render plane + dark_ball internally.
blend_file: data/agent_ball.blend
render_mode: ball
part_name: SolidModel
# One block for every chat/image model. Same three fields on each role.
models:
designer: {provider: openai_server, model_id: gpt-6-astra, api_base: "https://api.openai.com/v1"}
classifier: {provider: openai_server, model_id: gpt-6-astra, api_base: "https://api.openai.com/v1"}
critic: {provider: openai_server, model_id: gpt-6-astra, api_base: "https://api.openai.com/v1"}
inspector: {provider: openai_server, model_id: gpt-6-astra, api_base: "https://api.openai.com/v1"}
prompt_enricher: {provider: openai_server, model_id: gpt-6-astra, api_base: "https://api.openai.com/v1"}
texture_generator: {provider: openai_server, model_id: gpt-image-2.5-flare, api_base: "https://api.openai.com/v1"}
image_metrics:
model_path: Qwen/Qwen3-VL-Reranker-8B
adapter_path: yuanze1024/ShaderAgent-MatScorer-Qwen3VLReranker-LoRA
device: cuda
dtype: bfloat16
merge_lora: true
param_tuning:
max_iter: 50 # numeric search steps per structure step
max_params: 5
seed: 42
accept_prob: 0.05 # chance to keep a worse candidate
momentum: 0.0
log_range: 2.3 # log-uniform scale: exp(±log_range)
color_sv_perturb: 0.2 # HSV S/V step
hue_perturb: 0.15 # 0 keeps hue fixed
bool_flip_prob: 0.2
render_samples: 256
render_resolution: 256
critic:
good_enough_min: 0.8 # winner must clear this on every rubric dimension to stop
职责拆分具体而有效。执行错误修补、参数探索、结构改写各有明确反馈对象,避免让一个模型反复“看图再试”。DSL 是务实的工程接口。作者没有把它包装成全新的材质表示;类型图为检查、序列化和局部编辑提供统一入口。评分器面向真正的内环瓶颈。粗类别与细差异课程训练有相应的评价实验,且公开了权重和标注。
单视图逆问题没有消失。把天空反光拟合成 albedo 可以获得看似匹配的球,却无法正确换光;几何决定的纤维、薄膜和轮廓也不能靠 shader 完全补齐。评价规模和协议不够透明。45 例、扩展材质过采样、缺少置信区间与完整人评协议,限制对广泛生产任务的外推。
训练目标与评测耦合。MatScorer 的提升可能包含针对自身偏差的优化;需要独立人评、未知光照与编辑任务。局部搜索效率有限。固定最多 5 项参数、宽幅扰动、三视图重复渲染仍可能很贵。没有等成本曲线、渲染总时长或 GPU 显存统计时,不能把调用数优势直接外推为规模化优势。
editability 主要由表示与示例支持。节点可打开和可改,并不自动意味着图简洁、艺术家易理解或编辑响应解耦。缺少固定编辑任务、人工耗时、节点复杂度与副作用评测。强依赖外部模型与场景。闭源 VLM 版本变化、手工节点清单、预设灯光、prompt 偏好都影响结果。
Author Claim:这是通向可规模化生产材质创作的一步。Our Interpretation:论文证明了一个有限样本上的可运行闭环原型与偏好改善,尚未验证生产级稳定性、广泛资产兼容、真实物理恢复或全自动场景材质化。
以下全部是研究建议,不属于论文已经证明的贡献。优先围绕可测量的失败与成本提出问题。
| 方向 | 可实施方法 | 必须设置的验证 |
|---|---|---|
| 坐标与评测协议先对齐 | 统一像素/归一化框;发布数据划分与 judge 配置;锁定模型版本 | 同 checkpoint、同输入比较修复前后 ROI 与评分;建立独立 holdout |
| 执行感知、物理范围感知的搜索 | 从 Blender socket metadata 读取参数范围,区分颜色/向量;对正数用 log 变换 | 固定候选次数与 wall-time 比较成功率、边界命中率和人评 |
| 自适应结构/参数预算 | 用最近数轮增益、Critic 不确定性与搜索方差决定继续调参还是改拓扑 | 对固定 3×50 基线;报告质量-时间 Pareto 曲线,避免只报最好样本 |
| 多视图一致的 Critic | 三视图与新光照组合;显式判断发光、透射和 albedo/illumination 混淆 | 留出未知 HDRI、目标几何与角度,防止只适配预览材质球 |
| 可校准的细粒度 reward | 增加困难样本、多标注者一致性、rank group weighting 与置信区间 | 测试 material-disjoint / scene-disjoint / source-disjoint 泛化及 search reward hacking |
| 可编辑性 benchmark | 给定修改目标如“粗糙度提高但纹理尺度不变”;评价解耦与节点复杂度 | 编辑前后多条件渲染、人类耗时、局部改动量、非目标属性漂移 |
| 几何—材质联合表示 | 将纤维/薄膜失败转交几何模块,保留 shader 的局部光学逻辑 | 与纯 shader、完整几何生成比较成本及 silhouette / shading 准确性 |
若要迁移到 Substance/material graph 项目,最可复用的是“类型 IR+真实执行反馈+结构与参数分层搜索”的组织方式。需要重新定义节点签名、renderer 接口、参数域、合法性与偏好数据,不能直接假设 Blender MatScorer 或 node DSL 会无损迁移。
核查日期:2026-09-26。图 1–6 为原论文图像裁切,表 1–3 为数值重建;中文解释与 reviewer 分析为本次精读整理。引用页码按 PDF 的物理页码。
1. ShaderAgent 原论文 · 固定 release v1.0
2. 官方仓库 · 固定提交 ee84ce7b0304873024e2178a78704884e0861659
3. MatScorer 模型卡 · 固定 revision
4. MatScorer 数据卡与标注 · 固定 revision
8. Critic 实现
缺失材料:独立 supplementary、完整 MatScorer 训练 recipe、45 项主评测复现协议、正式出版元数据确认。没有将这些缺项用推测填补。