Model Serving

Chutes: a serverless computing platform for deploying and scaling open source AI models

Chutes is an AI model computing platform for developers. It is based on a decentralized open source architecture and users do not need to manage complex servers themselves. Using this platform, developers can quickly deploy and run various open source AI models, such as large language models or image generation models. Ch...

vLLM CLI: Command Line Tool for Deploying Large Language Models with vLLM

vllm-cli is a command line interface tool for vLLM that makes deploying and managing large language models much easier. The tool provides both an interactive menu interface and a traditional command line mode. It allows users to manage local and remote models, use preset or customized configuration schemes,...

LMCache: A Key-Value Cache Optimization Tool for Accelerating Reasoning on Large Language Models

LMCache is an open source key-value (KV) cache optimization tool designed to improve the efficiency of Large Language Model (LLM) reasoning. It significantly reduces inference time and GPU resource consumption by caching and reusing the intermediate computation results (key-value caching) of the model, which is especially suitable for long context scenarios.LMCache works with vL...

FastDeploy: an open source tool for rapid deployment of AI models

FastDeploy is an open source tool developed by the PaddlePaddle team that focuses on rapid deployment of deep learning models. It supports a wide range of hardware and frameworks, covering more than 20 scenarios such as image, video, text and speech, and contains more than 150 mainstream models.FastDeploy provides production environment out-of-the-box ....

Web - macOS AI Browser: a native AI-powered browser for macOS

Web is an open source macOS browser project developed by nuance-dev and hosted on GitHub. It is based on Apple's WebKit engine, using the SwiftUI and Combine frameworks, and follows the MVVM architecture.The core feature of Web is the set of ...

Transformers: open source machine learning modeling framework with support for text, image and multimodal tasks

Transformers is an open source machine learning framework developed by Hugging Face focused on providing advanced model definitions to support inference and training for text, image, audio, and multimodal tasks. It simplifies the process of using models and is compatible with many mainstream deep learning frameworks such as PyTorch, Tens .....

Hyperspace (aiOS): distributed AI arithmetic sharing network, aiOS generative browser, deep knowledge intelligences

Hyperspace is an innovative generative browser (aiOS), based on the world's largest peer-to-peer AI network, designed to provide users with powerful tools for deep research and analysis. By integrating a wide range of AI models and data sources, Hyperspace allows users to quickly generate information nets, utilizing high-quality sources such as Wikiped...

RunPod: GPU Cloud Service Designed for AI with Fast Cold Start SD and Pay Per Second

RunPod is a cloud computing platform designed specifically for AI, aiming to provide developers, researchers and enterprises with a one-stop solution for AI model development, training and scaling. The platform integrates on-demand GPU resources, serverless reasoning, and automatic scaling to provide powerful support for all stages of AI projects....

OpenBayes: Rapid Deployment of Rich Large Model Instances Using Cloud Computing Resources

OpenBayes is an out-of-the-box Artificial Intelligence and High Performance Computing (AI+HPC) service platform for machine learning engineers, providing multi-version framework support and rich datasets. Based on JupyterLab, it supports containerization and Kubernetes resource scheduling. At the same time, open a variety of APIs and private deployment options...

Range Rover Starship: Providing an Integrated Platform for GPU Arithmetic and AI Training and Reasoning in the Cloud

Lanrui Starship (Lanrui-ai) is a platform that provides cloud-based AIGC (Artificial Intelligence Generated Content) and AI training and pushing integrated arithmetic. The platform is independently developed by Wingsquare and aims to provide users with cost-effective AI arithmetic solutions. Rangefinder Starship integrates a variety of AI toolchains and supports multi-machine and multi-card distributed...

Ollama: Native One-Click Deployment of Open Source Large Language Models

Ollama General Description ollama is a lightweight native language model runtime framework that allows users to easily build and run large language models. It offers multiple quick start and installation options, supports Docker, and includes a rich set of libraries for users to choose from. It is easy to use, provides REST ap...