🔍 Read the full analysis: What Does It Take To Fine-Tune 3D Generation Models? on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Hugging Face published scripts and containerized workflows for fine-tuning its TRELLIS.2 3D-generation model on custom asset datasets, locally or through Hugging Face Jobs. The example uses LoRA adapters and 188 footwear objects to target shoe geometry, but the supplied account reports no completed evaluation or measured improvement.
Hugging Face has published scripts and containers for fine-tuning its TRELLIS.2 3D-generation model on custom asset datasets, with runs available on a user’s own machine or through the company’s cloud Jobs service. The accompanying example targets footwear geometry using LoRA adapters and 188 shoes from Google Scanned Objects, but the post provides no final results showing whether the tuning improved generated shoes.
The example is aimed at a specific flaw observed in the authors’ initial tests: TRELLIS.2 could generate shoes that looked plausible, but the soles were inaccurate. The stated goal is to test whether adapting the model to footwear can improve that part of the geometry. The supplied material describes the experiment’s dataset and training choices, not a completed comparison or a measured outcome.
The workflow uses low-rank adaptation, or LoRA, through the PEFT library. LoRA adds relatively small trainable adapter matrices while keeping the model’s original weights frozen. Hugging Face says this reduces the number of parameters being trained and the memory required for gradients and optimizer state; the post does not quantify those savings for this run.
The example places adapters on TRELLIS.2’s sparse-structure and shape-generation components, which influence the object’s form and detailed geometry. It leaves the texture-generation stage unchanged. The post says the shoe assets include diffuse color maps but not the separate material maps needed to train that stage, and that the base model’s colors and textures were already satisfactory for this task.
Fine-tuning TRELLIS.2 for custom 3D assets
Hugging Face released scripts and containers to adapt TRELLIS.2 to specific object categories. The footwear example outlines an experiment, but reports no completed results.
What the workflow targets
The example addresses shoes that look plausible but have inaccurate soles.
Coarse shape
LoRA adapters target sparse structure to adapt the model to footwear.
Detailed shape
The shape stage generates geometry; the example aims to improve shoe details such as soles.
Stage left unchanged
The dataset lacks separate material maps, and the authors say base colors and textures are satisfactory.
How TRELLIS.2 builds a mesh
The described pipeline turns an input image into a textured mesh.
BiRefNet removes the background, then DINOv3-L encodes the image.
Sparse structure guides the model’s detailed geometry generation.
A separate texture stage generates materials before the outputs are combined into a GLB mesh.
Ways to run the experiment
Teams with suitable assets can choose a local or cloud workflow.
- Use the provided Docker commands with your own GPU.
- Run preprocessing and training scripts on your machine.
- Run containerized workloads on cloud hardware.
- Upload the mesh dataset to the Hugging Face Hub.
What teams need
Access and evaluation matter alongside the training setup.
- A Hugging Face token and access to gated DINOv3.
- A comparison of pretrained and fine-tuned outputs, including held-out footwear.
- Clear geometry criteria and reported training settings, hardware, and costs.
A Route to Category-Specific 3D
The release offers developers a practical way to test whether a general-purpose 3D generator can be adapted to the demands of a narrower category. In applications such as robotics, simulation and media production, visual plausibility may not be enough: a model can look like a shoe while still having geometry that is unsuitable for a downstream task. A targeted training workflow gives teams a way to investigate category-specific shortcomings rather than relying only on the pretrained model.
Packaging preprocessing and training in scripts and containers may also make it easier to reproduce experiments across local hardware and cloud runs. The practical value remains unproven in this example, however. The material does not establish whether the tuned model produces more accurate soles, whether any gains generalize beyond the training assets, or how much preparation and computing capacity the process requires.
Products Worth Considering
Kindle Paperwhite Signature Edition 32GB E-Reader, 7-Inch Display, Seaglass Green
Bose QuietComfort True Wireless Earbuds with Active Noise Cancellation, Bluetooth 5.3, USB-C Charging, and IPX4 Water Resistance
Jackery Explorer 2000 v2 Portable Power Station with Two 200W Solar Panels
As an affiliate, we earn on qualifying purchases.
How TRELLIS.2 Produces a Mesh
TRELLIS.2 uses a staged pipeline to turn an input image into a textured 3D mesh. As described in the source post, it first removes the image background with BiRefNet and encodes the image with DINOv3-L. A sparse-structure model predicts a coarse shape; a shape model then generates detailed geometry, and a separate texture model creates surface materials. The outputs are combined into a mesh that can be exported as a GLB file.
The post presents TRELLIS.2 as an extension of TRELLIS’s approach of using a shared latent representation that can support different 3D outputs through decoders. For the footwear experiment, the selected components reflect the specific target: shape and structure affect the soles, while texture is left alone. That rationale explains the training plan, but does not show that this component selection is better than alternatives.
The Shoe Test Has No Results
The supplied account does not say whether the experiment was completed or whether sole geometry improved. It includes no before-and-after examples, quantitative benchmark or independent evaluation. It also does not specify the test set, scoring criteria, or how the authors judged whether a sole was accurate.
Other practical details are absent, including the hardware configuration, training time, cost and amount of dataset preparation. The post says the scripts support local execution and Hugging Face Jobs, but does not report reproducibility results or performance on footwear outside the 188-object subset. The stated concern about soles comes from the authors’ initial tests, whose full setup is not described in the supplied material.
Run the Workflow and Test Generalization
Developers can adapt the published scripts after uploading a mesh dataset to the Hugging Face Hub, then run preprocessing and training locally in Docker or through Hugging Face Jobs. The workflow requires a Hugging Face token and access to the gated DINOv3 model used to encode input images.
The next evidence needed is an evaluation comparing the pretrained and fine-tuned models on defined footwear test images, including examples not used for training. Such reporting could show whether sole geometry improves, whether changes affect other parts of the mesh, and what compute and preparation the process demands. The supplied source does not give a date for those results or say when they may be published.
Key Questions
What did Hugging Face release?
Scripts and containers for fine-tuning TRELLIS.2 on custom 3D assets, with workflows for local machines and Hugging Face Jobs.
What does the footwear example try to improve?
It tests whether fine-tuning can improve shoe geometry, especially soles, which the authors said were inaccurate in initial tests.
What data and method does the example use?
The example uses 188 footwear objects from Google Scanned Objects and LoRA adapters applied to the sparse-structure and shape-generation components.
Did fine-tuning improve the generated shoes?
No result is reported in the supplied material. It provides no completed comparison, benchmark or before-and-after examples.
What do users need to run the workflow?
Users need a prepared mesh dataset, a Hugging Face token, access to the gated DINOv3 model and either suitable local GPU hardware or a Hugging Face Jobs run.
Primary source: Hugging Face · via ThorstenMeyerAI.com
