gpt-oss-space-game: a local voice-interactive space game built using open-source AI models

gpt-oss-space-game is an open source space-themed game project hosted on GitHub and created by developer kwindla. It combines OpenAI's open-source language model gpt-oss, the speech recognition technology Whisper, the speech synthesis technology Kokoro TTS and the Pipecat framework to build a voice-interactive game that runs locally. Players experience an immersive space adventure by talking to the AI characters in the game through voice. The project is designed to be flexible, allowing developers to replace components or connect to cloud services, making it suitable for local operation and customized development. The entire project is free and open under the Apache 2.0 license, and developers are encouraged to contribute and experiment.

Function List

• voice interaction: Players control the gameplay by having real-time conversations with the game's AI characters through a microphone.
• local operation: The game supports running on local devices without relying on cloud services, protecting data privacy.
• modular design: Support for replacing components such as gpt-oss, Whisper, etc. to adapt to cloud or self-hosted APIs.
• Open Source Modeling Support: Optimizing low-latency and local inference using OpenAI's gpt-oss-20b or gpt-oss-120b models.
• Flexible Configuration: Developers can adjust the level of inference of AI models to accommodate different hardware performance.
• cross-platform compatibility: Supports running on multiple operating systems, suitable for developers to debug and deploy.

Using Help

Installation process

To run gpt-oss-space-game, you need to configure the environment and install dependencies on the local device. The following detailed installation steps are for users familiar with command line operations.

1. Cloning project warehouse

First, use Git to clone the project locally:

git clone https://github.com/kwindla/gpt-oss-space-game.git
cd gpt-oss-space-game

Make sure Git is installed. If not, you can download and install it from the Git website.

2. Installation of dependencies

The project relies on the Python environment and several open source components. Python 3.11 or later is recommended. Here are the steps for installing the dependencies:

• Installing Python: Download and install Python 3.11+ from the Python website, making sure to add Python to the system PATH.
• Install Pipecat: Pipecat is the core framework for voice interaction, run the following command to install it:

  pip install pipecat
  

• Install Whisper: Whisper is used for speech recognition, installation method:

  pip install openai-whisper
  

  Make sure that FFmpeg (a dependency of Whisper) is installed for Windows users via a package manager (e.g. Chocolatey) and running for Linux users:

  sudo apt-get install ffmpeg
  

• Installation of Kokoro TTS: for speech synthesis, installation commands:

  pip install kokoro-tts
  

• Installing the gpt-oss model: The project supports the gpt-oss-20b and gpt-oss-120b models. It is recommended to use llama.cpp of the llama-server runtime model. Download llama.cpp:

  git clone https://github.com/ggerganov/llama.cpp.git
  cd llama.cpp
  make
  

  Then download gpt-oss Model files (found at Hugging Face, for example) ggml-org/gpt-oss-20b-GGUF maybe ggml-org/gpt-oss-120b-GGUF).

3. Configuration model reasoning

The project uses local inference by default, you need to configure the inference level of the gpt-oss model to be "low" to optimize the voice interaction. Run llama-server:

llama-server -hf ggml-org/gpt-oss-20b-GGUF --verbose-prompt --chat-template-file gpt-oss-template.jinja --jinja --cache-reuse 128 -fa

• -hf Specifies the model file path.
• --chat-template-file Use the project-supplied gpt-oss-template.jinja document that sets the level of reasoning as low.
• --cache-reuse 128 Optimize memory usage.
  If using gpt-oss-120b, make sure the device has at least 80GB of GPU memory; gpt-oss-20b requires 16GB.

4. Running the game

After all dependencies are installed, run them in the project directory:

python main.py

When the game starts, the system will load the voice interaction module. Make sure the microphone is connected and working properly.

Operation of the main functions

voice interaction

• Starting a dialogOnce the game starts, the player speaks commands over the microphone, such as "start ship" or "navigate to Mars". whisper converts the speech to text, gpt-oss processes the commands and generates a response, and Kokoro TTS converts the response into a Kokoro TTS converts the response into voice output.
• Control games: A variety of commands are supported, such as "view star map", "fire weapon", etc. The specific command set can be found in the project documentation. The command set can be found in the project documentation.
• Real-time feedback: The AI responds instantly to player commands, such as describing the space environment or performing a mission.

Customized Configuration

• Replacement cloud services: Developers can modify API endpoints in code to connect to cloud gpt-oss or other models. For example, editing the config.py file, replacing the local model path with the cloud API address.
• Adjusting the level of reasoning: if higher reasoning skills are required, modify the gpt-oss-template.jinjaIf the reasoning level is set to "medium" or "high", it requires stronger hardware.
• Add New Feature: The project supports the expansion of new modules, such as adding new voice commands or game scenarios, which can be edited by the developer. main.py and related scripts.

caveat

• hardware requirement: gpt-oss-20b is suitable for normal PCs (16GB RAM), gpt-oss-120b requires a high performance GPU (e.g. NVIDIA H100).
• adjust components during testing: If you encounter a model loading failure, check if the GPU memory is sufficient or enable the PYTORCH_CUDA_ALLOC_CONF=expandable_segments:TrueThe
• documentation support: The project README and GitHub pages provide detailed configuration instructions, which are recommended to be read carefully.

application scenario

1. AI Game Development Experiment
   Developers can learn how to integrate open source AI models into games with gpt-oss-space-game. The project demonstrates the synergy of speech recognition, language modeling, and speech synthesis, making it suitable for investigating the potential of AI in interactive applications.
2. Local Privacy Protection Game
   Users want to play the game in a network-free environment while protecting data privacy. The project runs completely locally, and voice and text data are not uploaded to the cloud, making it suitable for privacy-sensitive users.
3. Education and Technology Presentations
   Schools or tech communities can use the program to demonstrate AI technology. Students learn about speech processing, AI reasoning, and open source software development by building and modifying games.
4. Game prototyping
   Independent developers can develop new games based on this program. The modular design allows to add new scenes, characters or interactions and quickly build prototypes of space adventure games.

QA

1. Does the game require an internet connection?
   Not required. gpt-oss-space-game supports fully local operation, with all modeling and processing done locally. However, developers can choose to connect to the cloud API to replace the local model.
2. What languages are supported?
   Currently, it mainly supports English voice interaction, Chinese support requires additional configuration. Whisper and Kokoro TTS in Chinese. Developers can replace the model files in their projects.
3. How do I contribute code?
   fork the project on GitHub, modify the code and submit a pull request. it is recommended to read the CONTRIBUTING.md file first to understand the code specification.
4. What if the model fails to load?
   Check that the hardware meets the requirements (gpt-oss-20b requires 16GB of RAM, gpt-oss-120b requires 80GB GPU). Make sure the model file path is correct, or check the logs for error messages.