LLM2Jev
Turn local language models into Jev-style structured decision models. Get results from text and images with prefill alone—no token-by-token decoding required.
What is LLM2Jev?
Turn local language models into Jev-style structured decision models. Get results from text and images with prefill alone—no token-by-token decoding required.
Overview of LLM2Jev
What it is
LLM2Jev is an open-source tool that turns local language models (LLMs) and multimodal language models (MLLMs) into Jev-style structured decision models. It allows users to obtain decision probabilities from text and images using prefill alone, without token-by-token decoding. The project supports SGLang, Transformers, and MLX backends, and provides a Python client built on the OpenAI SDK.
Key features
- No training or fine-tuning required; uses existing model weights with SGLang, Transformers, or MLX on Apple Silicon.
- Chat and decisions share the same service and model when using the SGLang backend.
- Chat and decisions share a client:
ClientandAsyncClientextend the OpenAI Python SDK withsystemone.create(). - Prefill-only scoring:
Choice,Score, andNoulread yes/no logits without generating answers;Boxgenerates normalized image coordinates. - Supports text and image decisions by combining them in
stateorinstructions. - Four question types in one request: mix
Choice,Score,Noul, andBoxto get option probabilities, rubric scores, yes/no probabilities, and image bounding boxes. - Reuse shared prefixes to reduce repeated computation across candidates via SGLang's Radix Cache or MLX's explicit prefix prefill.
Use cases
- Classification, scoring, and action selection from text and images.
- Building decision engines for customer support routing, content moderation, or game AI.
- Combining chat and structured decisions in a single service for applications that need both conversational and deterministic outputs.
Getting started
On Linux with a supported NVIDIA GPU, start one SGLang service for chat and decisions: bash git clone https://github.com/Yinsongxu/LLM2Jev.git cd LLM2Jev uv sync --extra sglang source .venv/bin/activate llm2jev-serve --backend sglang --model-path /path/to/model --served-model-name local-model --host 127.0.0.1 --port 30000 Replace /path/to/model with a model directory supported by SGLang. Then use the Python client to call both endpoints. See the repository README for full installation and usage details.
License and pricing
Licensed under the Apache License 2.0. The project is open source.
Based on the project's public README and repository metadata.