LLM2Jev

Turn local language models into Jev-style structured decision models. Get results from text and images with prefill alone—no token-by-token decoding required.

What is LLM2Jev?

Turn local language models into Jev-style structured decision models. Get results from text and images with prefill alone—no token-by-token decoding required.

Overview of LLM2Jev

What it is

LLM2Jev is an open-source tool that turns local language models (LLMs) and multimodal language models (MLLMs) into Jev-style structured decision models. It allows users to obtain decision probabilities from text and images using prefill alone, without token-by-token decoding. The project supports SGLang, Transformers, and MLX backends, and provides a Python client built on the OpenAI SDK.

Key features

  • No training or fine-tuning required; uses existing model weights with SGLang, Transformers, or MLX on Apple Silicon.
  • Chat and decisions share the same service and model when using the SGLang backend.
  • Chat and decisions share a client: Client and AsyncClient extend the OpenAI Python SDK with systemone.create().
  • Prefill-only scoring: Choice, Score, and Noul read yes/no logits without generating answers; Box generates normalized image coordinates.
  • Supports text and image decisions by combining them in state or instructions.
  • Four question types in one request: mix Choice, Score, Noul, and Box to get option probabilities, rubric scores, yes/no probabilities, and image bounding boxes.
  • Reuse shared prefixes to reduce repeated computation across candidates via SGLang's Radix Cache or MLX's explicit prefix prefill.

Use cases

  • Classification, scoring, and action selection from text and images.
  • Building decision engines for customer support routing, content moderation, or game AI.
  • Combining chat and structured decisions in a single service for applications that need both conversational and deterministic outputs.

Getting started

On Linux with a supported NVIDIA GPU, start one SGLang service for chat and decisions: bash git clone https://github.com/Yinsongxu/LLM2Jev.git cd LLM2Jev uv sync --extra sglang source .venv/bin/activate llm2jev-serve --backend sglang --model-path /path/to/model --served-model-name local-model --host 127.0.0.1 --port 30000 Replace /path/to/model with a model directory supported by SGLang. Then use the Python client to call both endpoints. See the repository README for full installation and usage details.

License and pricing

Licensed under the Apache License 2.0. The project is open source.

Based on the project's public README and repository metadata.

Add this badge to your README

Featured on YunoDB Badge styles & help

Maintain this project? Show that it is featured on YunoDB.

Markdown[![Featured on YunoDB](https://yunodb.com/badge/yinsongxullm2jev.svg)](https://yunodb.com/tools/yinsongxullm2jev?utm_source=github&utm_medium=badge)
HTML<a href="https://yunodb.com/tools/yinsongxullm2jev?utm_source=github&utm_medium=badge"><img src="https://yunodb.com/badge/yinsongxullm2jev.svg" alt="Featured on YunoDB" height="20"></a>