InstantIR: damaged image repair and image high-definition zoom open source project, minimum 16G video memory

General Introduction

InstantIR is an innovative single-image restoration model developed by the InstantX team, designed to resurrect your damaged images with extremely high-quality and lifelike details, enabling high-quality restoration of damaged images. The tool not only restores image details, but enhances the accuracy of the restored image with additional textual hints.InstantIR utilizes SDXL and DINOv2 models, providing flexible pipeline configurations that can be adapted by the user to meet specific needs.

Online experience: https://huggingface.co/InstantX/InstantIR

InstantIR Configuration

Function List

• Image Restoration: High-quality restoration of damaged or low-quality images.
• Generating Models: Image detail recovery using generative modeling techniques.
• Text Tip Editor: Customized editing of images through text prompts.
• Flexible Configuration: Supports a variety of parameter adjustments to meet different image processing needs.
• local deployment: Gradio scripts are provided to support local deployment and demos.
• compatibility: Compatible with diffusers and supports many powerful features.

Using Help

Installation process

1. Clone the repository and set up the environment::

   git clone https://github.com/instantX-research/InstantIR.git
   cd InstantIR
   conda create -n instantir python=3.9 -y
   conda activate instantir
   pip install -r requirements.txt
   

2. Download pre-trained model: InstantIR is based on the SDXL and DINOv2 models, which can be downloaded from HuggingFace:

   from huggingface_hub import hf_hub_download
   hf_hub_download(repo_id="stabilityai/stable-diffusion-xl-base-1.0")
   hf_hub_download(repo_id="facebook/dinov2-large")
   hf_hub_download(repo_id="InstantX/InstantIR")
   

3. running inference: Use infer.sh Scripts for reasoning:

   ./infer.sh --sdxl_path <path_to_SDXL> --vision_encoder_path <path_to_DINOv2> --instantir_path <path_to_InstantIR> --test_path <path_to_input> --out_path <path_to_output>
   

Tips for use

• excessive smoothing: Will --cfg The parameter is adjusted between 3.0 and 5.0.
• low fidelity: Settings --preview_start is 0.1 to 0.4 to maintain input fidelity.
• localized distortion: Will --creative_start Set to 0.6 to 0.8 to generate high-frequency detail in post.
• accelerated reasoning:: Improvement --preview_start and lower --creative_start It can reduce computational costs and speed up reasoning.

Using diffusers

InstantIR is fully compatible with diffusers and can be loaded and used directly:

import torch
from PIL import Image
from diffusers import DDPMScheduler
from schedulers.lcm_single_step_scheduler import LCMSingleStepScheduler
from module.ip_adapter.utils import load_adapter_to_pipe
from pipelines.sdxl_instantir import InstantIRPipeline
# 加载预训练模型
pipe = InstantIRPipeline.from_pretrained('stabilityai/stable-diffusion-xl-base-1.0', torch_dtype=torch.float16)
load_adapter_to_pipe(pipe, 'facebook/dinov2-large')
pipe.prepare_previewers('path_to_InstantIR')
pipe.scheduler = DDPMScheduler.from_pretrained('stabilityai/stable-diffusion-xl-base-1.0', subfolder="scheduler")
lcm_scheduler = LCMSingleStepScheduler.from_config(pipe.scheduler.config)
# 加载损坏图像并进行修复
low_quality_image = Image.open('path_to_image').convert("RGB")
image = pipe(image=low_quality_image, previewer_scheduler=lcm_scheduler).images[0]

Local Deployment Gradio Demo

A Python script is provided for local deployment of the Gradio demo:

INSTANTIR_PATH=<path_to_InstantIR> python gradio_demo/app.py

Then visit in your browser http://localhost:7860 Conduct a demonstration.

InstantIR One-Click Installer

Fix 1024 resolution images with at least 24gb memory and at least 16gb video memory, don't try it with low end graphics cards. After generating multiple pictures at the same time, memory recovery is abnormal.

This content has been hidden by the author, please enter the verification code to view the content

Captcha:

Please pay attention to this site WeChat public number, reply "CAPTCHA, a type of challenge-response test (computing)", get the verification code. Search in WeChat for "AI productivity applications"or"Artificial9527"or WeChat scanning the right side of the QR code can be concerned about this site WeChat public number.