PartCrafter: Generating Editable 3D Part Models from a Single Image

PartCrafter is an innovative open source project focused on generating editable 3D part models from a single RGB image. It uses advanced structured 3D generation techniques to simultaneously generate multiple semantically meaningful 3D parts from a single image, applicable to game development, product design and other fields. The project is based on a pre-trained 3D Mesh Diffusion Transformer (DiT), which introduces combinatorial latent space and hierarchical attention mechanisms to ensure that the generated 3D model maintains overall consistency while preserving part details.PartCrafter provides training data for 130,000 3D objects by integrating a large dataset of 3D objects, 100,000 of which contain multi-part annotations. The project plans to release inference scripts and pre-trained models by July 15, 2025, with a HuggingFace demo to facilitate user experience and development.

Function List

• Generate multiple editable 3D part models from a single RGB image.
• Support for generating 3D meshes for complex multi-object scenes.
• Provides combinatorial latent space to represent each 3D part independently.
• Hierarchical attention mechanisms are used to ensure global consistency and detail retention between parts.
• Compatible with large 3D datasets and supports part-level supervised training.
• Plans are to release inference scripts, pre-trained models, and HuggingFace demos.

Using Help

Installation process

As of June 2025, the complete code and pre-trained models for PartCrafter have not yet been fully released. Inference scripts and pre-training checkpoints are expected to be released by July 15, 2025, according to the official GitHub page. The following is a speculative installation and usage process based on current information, which may be adjusted in the future based on official updates.

1. environmental preparation
   PartCrafter is based on Python, Python 3.8 or higher is recommended. Users need to install the following dependencies (wait for the official release of the dependency list):

   pip install torch numpy opencv-python
   

   Ensure that the system supports GPU acceleration (e.g. CUDA) to improve model inference. A Linux or Windows system with at least 16GB of RAM and an NVIDIA GPU is recommended.

2. clone warehouse
   Visit the official GitHub repository https://github.com/wgsxm/PartCrafter, clone the project locally:

   git clone https://github.com/wgsxm/PartCrafter.git
   cd PartCrafter
   

3. Installation of pre-trained models
   The official plan is to release pre-training checkpoints. Users can download the checkpoint file and place it in a specified directory in the repository (e.g.checkpoints/). The exact path and download link need to wait for the official update.
4. Running inference scripts
   Once the inference script is published, users can run the model from the command line. For example, suppose the script is namedinfer.py, the operation may be as follows:

   python infer.py --image_path <input_image.jpg> --output_dir <output_folder>
   

   Among them.--image_pathSpecifies the input RGB image path.--output_dirSpecifies the path where the generated 3D model is saved.

Main Functions

The core function of PartCrafter is to generate editable 3D part models from a single RGB image. Below are the detailed steps:

1. Preparing to enter an image
   Users are required to provide a clear RGB image that contains the target object or scene. A picture resolution of 512 x 512 or higher is recommended to ensure that the model captures sufficient detail. For example, take a picture of a chair that contains parts such as the seat, backrest, and legs.
2. running inference
   Images are processed using inference scripts. The model will analyze the contents of the image to generate multiple 3D part models. For example, after inputting an image of a chair, PartCrafter may output separate 3D meshes for the seat, backrest, and four legs. The output format is usually an OBJ or PLY file, compatible with Blender, Unity, and other software.
3. Editing 3D Models
   The generated 3D model supports part-level editing. Users can open the model in Blender and adjust the geometry, texture, or position of individual parts. For example, modify the angle of a chair's backrest or the length of its legs without affecting other parts.
4. Multi-object scene generation
   For complex scenes (e.g. pictures containing tables and chairs), PartCrafter can generate part models for multiple objects simultaneously. The model ensures semantic consistency between parts through a hierarchical attention mechanism, e.g., stylistic uniformity between table legs and chair legs.
5. HuggingFace demo use
   Officials plan to launch a HuggingFace demo that will allow users to upload images and access 3D models through an online interface. Specific operations include:
   • Visit the PartCrafter page on the HuggingFace platform.
   • Upload the RGB image and set the generation parameters (e.g. number of parts or resolution).
   • Download the generated 3D model file.

Featured Function Operation

• combined latent space: Each 3D part consists of a set of independent latent tokens indicated that users can modify specific parts by adjusting tokens. For example, changing the tokens of a chair back can generate different shapes of backrests.
• Hierarchical Attention Mechanisms: The model automatically balances global consistency between parts and local details during generation. There is no need for the user to manually adjust part relationships; the model ensures that the generated chair legs are stylistically coordinated with the seat.
• parts-level supervision: Thanks to a dataset of 130,000 3D objects, PartCrafter can generate parts that are not directly visible in the image. For example, enter a picture of the front of a chair and the model can infer and generate the back part.

caveat

• Ensure that the input image is well lit and has a simple background to improve the quality of the generation.
• Generated 3D meshes are suitable for rendering and game development, but conversion to industrial-grade solid models may require additional processing (e.g., conversion using specialized software).
• Official documentation and community support (e.g. GitHub Issues) can provide further help.

application scenario

1. game development
   Developers can use Cognizant PartCrafter to quickly generate 3D models of game assets. For example, generating editable 3D models from a single prop image speeds up the level design process.
2. product design
   Designers can use PartCrafter to convert product concept drawings into editable 3D part models for rapid prototyping and modification.
3. Education and Research
   Researchers can use PartCrafter to generate complex 3D models for teaching 3D visualization or experimental validation.

QA

1. What formats are supported for 3D models generated by PartCrafter?
   Currently supports OBJ and PLY formats, compatible with Blender, Unity and other mainstream 3D software.
2. Do I need specialized knowledge to use PartCrafter?
   Not required, but users familiar with Python and 3D modeling software can use the tools more efficiently.
3. How complex a model can PartCrafter generate?
   It generates 3D models of multi-part objects and complex scenes, with the exact complexity depending on the quality of the input image and the model training data.
4. Is the project fully open source?
   The project is open source, inference scripts and training data will be released gradually, subject to official updates.