Iterative Prompt Engineering with LLMStudio

Creating the ideal prompt can be the key to transforming a less than average outcome into one that is remarkably relevant. Discovering the right prompt often involves numerous revisions and a method of trial and error. I felt the need to refer to my past attempts, modifications made to prompts, LLM models I tried and other LLM settings as well relative cost of these combinations. There are a few options to help with this : promptflow, langsmith, LLMStudio and others.

I tried LLMStudio and promptflow. This article is about LLMStudio.

If you are installing LLMStudio on Windows, use WSL. Here are the steps :

  • Create a folder on your machine.
  • Create .env file here with following content:
  • Enter wsl and create a new python environment with conda
(base) PS c:\code\llmstudio> wsl
(base) ash@DESKTOP:/mnt/d/code/llmstudio$ conda activate lmstd

LLMStudio runs on Bun. Bun is a javascript runtime like Node.

sudo apt-get install unzip
powershell -c "irm|iex"
  • Install Node v18, if you run into ReferenceError: Request is not defined    at Object.<anonymous> (/home/ash/miniconda3/lib/python3.11/site-packages/llmstudio/ui/node_modules/next/dist/server/web/spec-extension/request.js:28:27)
(base) ash@DESKTOP:/mnt/d/code/lmstudio$ nvm install 18
  • Start LLMStudio
(base) ash@DESKTOP:/mnt/d/code/lmstudio$ llmstudio server –ui
LLMStudio is available at http://localhost:3000
You can select the LLM model you want to work with from the drop-down
Specify LLM parameter settings

Export the execution data as csv file by clicking on [Export data] button. This data includes the input, output, LLM model, Input and Output tokens as well as cost.

What is LLMstudio?
LLMstudio by TensorOps
LLM Studio Quickstart
Install Bun for Windows
Setup Bun JS in windows using WSL and VS code
NextJS – ReferenceError: Request is not defined