Student Project

Mic & Magic - Live AI assistant for Pen&Paper game masters

Staff

Brief Description

You speak - the AI listens, understands and inspires. Mic & Magic is an innovative practical project that combines the magic of pen & paper role-playing games with state-of-the-art voice AI. The result is a system that transcribes spoken dialogues in real time, recognizes speakers and generates creative suggestions for the game master - without any manual input.

Learning Objectives and Disciplines

The project aims to develop a game management assistance system that:

  • Records live audio from the game session,
  • Automatically recognizes speakers,
  • creates transcriptions in real time,
  • and continuously generates inspiring suggestions for small plots, encounters or twists from the ongoing conversation - based on the rules, world description and previous notes.

The AI acts like a silent co-player, constantly whispering in ideas.

Project Workflow

Phase 1 - Understanding

  • Integration of Whisper/WhisperX for live transcription
  • Speaker recognition with diarization and voice embeddings
  • Assignment of voices to defined characters (embedding cache)

Phase 2 - Thinking

  • Connection of a local LLM (e.g. via Ollama)
  • RAG system with world description, rules, notes
  • Continuous prompt generation for creative impulses

Phase 3 - Support

  • Development of a CLI application for the game management
  • Output of suggestions, character hints, tension building etc.

Bonus phase - Present

  • Optional desktop app to visualize transcript, speaker history, session summaries and AI ideas

Technologies & Tools

  • Audio & Diarization: Whisper, WhisperX, Insanely-Fast-Whisper, pyannote
  • Speech embedding: torchaudio, FAISS/Embedding Cache
  • LLM & RAG: Ollama, LangChain or own pipeline
  • Interface: Python (CLI), optional Tauri/Electron for GUI
  • Audio input: Microphone or combined stream via VoiceMeeter

Who is the project suitable for?

The project is ideal for students who…

  • are interested in practical AI development
  • are interested in audio processing, NLP or role-playing systems
  • like to think across systems (audio, embeddings, LLM, CLI/UI)
  • want to work creatively with technology

Previous knowledge of Python is helpful, but not a prerequisite - having fun with AI, game design and systemic thinking is more important!

Kick-Off Meeting

If you want to participate in the project, you are required to attend the kick-off meeting on 16.10.2025 at 10:00 in room BC 303. If that is not possible, message us before the meeting.

Material

Moodle course

To join the course you are required to enter a key which will be distributed during the kick-off meeting.