Migrate from vLLM

This document presents a guide for migrating from the vLLM API to the MK1 Flywheel API. This guide will help you understand the differences between the two APIs and how to update your code to use the MK1 Flywheel API.

The table below shows the mapping between the supported vLLM parameters and the MK1 Flywheel parameters.

vLLM	MK1 Flywheel	Comments
prompt	text
max_tokens	max_tokens
n	num_samples
stop_token_ids	eos_token_ids
ignore_eos		You can replicate behavior by setting eos_token_ids to an empty list.
stop	stop
temperature	temperature
top_k	top_k
top_p	top_p
presence_penalty	presence_penalty
frequency_penalty	frequency_penalty
skip_special_tokens		Behavior enabled by default.