Author: tewari

Iterative Prompt Engineering with LLMStudio

Creating the ideal prompt can be the key to transforming a less than average outcome into one that is remarkably relevant. Discovering the right prompt often involves numerous revisions and a method of trial and error. I felt the need to refer to my past attempts, modifications made to prompts, LLM models I tried and other LLM settings as well relative cost of these combinations. There are a few options to help with this : promptflow, langsmith, LLMStudio and others.

I tried LLMStudio and promptflow. This article is about LLMStudio.

If you are installing LLMStudio on Windows, use WSL. Here are the steps :

  • Create a folder on your machine.
  • Create .env file here with following content: 
OPENAI_API_KEY="sk-api_key"
ANTHROPIC_API_KEY="sk-api_key"
  • Enter wsl and create a new python environment with conda
(base) PS c:\code\llmstudio> wsl
(base) ash@DESKTOP:/mnt/d/code/llmstudio$ conda activate lmstd

LLMStudio runs on Bun. Bun is a javascript runtime like Node.

sudo apt-get install unzip
powershell -c "irm bun.sh/install.ps1|iex"
  • Install Node v18, if you run into ReferenceError: Request is not defined    at Object.<anonymous> (/home/ash/miniconda3/lib/python3.11/site-packages/llmstudio/ui/node_modules/next/dist/server/web/spec-extension/request.js:28:27)
(base) ash@DESKTOP:/mnt/d/code/lmstudio$ nvm install 18
  • Start LLMStudio
(base) ash@DESKTOP:/mnt/d/code/lmstudio$ llmstudio server –ui
LLMStudio is available at http://localhost:3000
You can select the LLM model you want to work with from the drop-down
Specify LLM parameter settings

Export the execution data as csv file by clicking on [Export data] button. This data includes the input, output, LLM model, Input and Output tokens as well as cost.

References
What is LLMstudio?
LLMstudio by TensorOps
LLM Studio Quickstart
Install Bun for Windows
Setup Bun JS in windows using WSL and VS code
NextJS – ReferenceError: Request is not defined

Prompt Engineering and Security – Custom GPT

I have been delving into Advanced Prompt Engineering and Security techniques for Large Language Models (LLMs). As an exercise, I have created a custom GPT in ChatGPT to help practice spelling of English words. Spell It GPT is secured with Advanced Prompt Engineering techniques to guard against common attack vectors, including Direct Prompt Injection, Prompt Leaking, Role Playing, Simulation, DAN and Code Injection.

Spell-It-GPT

Play with the Spell It GPT and try to break it. It is not impossible but (probably) fairly difficult to do 😉 Regardless, practice spelling and have fun!

Switch to Voice Mode in ChatGPT Mobile App to practice spelling and put headsets on for best results!

Spell It: https://bit.ly/3Ua0oib

Here are some resources you can learn more about Prompt Engineering and Security:
Advanced Prompt Engineering Techniques
OWASP Top 10 for Large Language Model Applications
Educational Resources
Adversarial Prompting in LLMs

Get an NFT at All Things Open 2022

You can get a free commemorative NFT if you met me at All Things Open Conference in Raleigh, NC.

  • Enter your Ethereum address (or ENS/Email)
Mint POAP NFT
  • Check [Free minting on Gnosis] for free minting of NFT on xDAI chain
    • If you uncheck this box you can migrate to Ethereum mainnet now but you will have to pay for that transaction. You can also migrate to mainnet later if you wish.
  • Click on Mint POAP button

Your NFT will be confirmed after the minting is complete.

POAP NFT Confirmation

POAPs are Non Fungible Tokens (NFT) issued on the blockchain. Learn more of POAPs here : https://poap.xyz/

Security Analysis of Etheruem Smart Contracts with Mythril

Mythril is an open-source security analysis tool for EVM bytecode, courtesy of ConsenSys. It is also a component of their Security Analysis Service – Mythx. Mythril detects security vulnerabilities in smart contracts built for Ethereum and other EVM-compatible blockchains.

Vulnerabilities found by Mythril are reported with reference to the weaknesses listed on the Smart Contract Weakness Classification Registry (SWC Registry). I will use two entries from SWC Registry for the examples in this article:

  • SWC-106 – Due to missing or insufficient access controls, malicious parties can self-destruct the contract.
  • SWC-107 – One of the major dangers of calling external contracts is that they can take over the control flow. In the reentrancy attack (a.k.a. recursive call attack), a malicious contract calls back into the calling contract before the first invocation of the function is finished.

Install Mythril on Windows

> docker import mythril/myth

https://mythril-classic.readthedocs.io/en/master/installation.html

Get test files from github

Source code for these tests is on github : mythril-tests. Clone the repo locally and adjust the paths in the commands below to match your local environment.

Analyze a local smart contract

Analysis of SelfDestructMultiTxFeasible.sol

> docker run -v E:\share\:/data mythril/myth -v4 analyze /data/mythx-tests\05222022-25/SelfDestructMultiTxFeasible.sol

Mythril reports an instance of SWC-106 vulnerability:


Analysis of SimpleDAO.sol

> docker run -v E:\share\:/data mythril/myth -v4 analyze /data/mythx-tests\05222022-25/SimpleDAO.sol

Mythril reports three instances of SWC-107 and one instance of SWC-105:

Analysis of a flatenned contract file

File containing the two test contracts returns five instances of vulnerabilities of both contracts:

> docker run -v E:\share\:/data mythril/myth -v4 analyze /data/mythx-tests\05222022-25/flatenned-01.sol

Analyze a contract with imported contract

Most smart contracts import other contracts to reuse functionality. You do not have to flatten the contracts into one file. Mythril can work with contracts with imports specified in them : SimpleDAOWithImport.sol

> docker run -v E:\share\:/data mythril/myth -v4 analyze /data/mythx-tests\05222022-26/SimpleDAOWithImport.sol

Analyze a contract with @OpenZeppelin style import

Mythril relies on solc for compiling contract source code. For @OpenZeppelin style imports, you have to specify –solc-json file containing remapping for solc to locate the referenced files : SimpleDAOWith-OzImport.sol

> docker run -v E:\share\:/data mythril/myth -v4 analyze /data/mythx-tests\05222022-26/SimpleDAOWith-OzImport.sol –solc-json=/data/solc-args.json

Analyzing On-Chain Contracts

Mythril can analyze contracts deployed on the blockchain directly. You do not need source code of the contract. Support for infura is built-in, you can also use custom RPC endpoint. Replace INFURA_ID with your Infura project id and CONTACT_ADDRESS with the address of your contract on the blockchain :

> docker run mythril/myth -v4 analyze –rpc infura-rinkeby –infura-id INFURA_ID -a CONTACT_ADDRESS

KHEL Coin on Ethereum

KhelCoin (KHEL) is an ERC-20 token available on Ethereum blockchain. It is a Smart Contract created with Solidity, OpenZeppelin, Truffle, Ganache, Remix, web3.js and VSCode. It is currently available on Rinkeby test network. It will be launched on the Mainnet in the near future.

You can get KHEL coins from this website – KHEL Coin ICO

%KhelCoin The Game Coin Metamask is needed to view your balance or buy KHEL Coins Click here to download Metamask METAMASK

Head over to the ICO website. You will be prompted to install Metamask and connect to your account on Rinkeby testnet. Click on “Buy KHEL Coins” button to get your own coins!

Source code is on github – https://github.com/ashtewari/khelcoin

Puzzle Icon Credit: https://www.flaticon.com/free-icons/puzzle

Kusto Queries on AKS Clusters

Kusto query language can be used to get insights into Azure Kubernetes Service (AKS) clusters. Container insights collects data from AKS clusters and forwards it to Log Analytics workspace, if enabled for a cluster. This data is available for querying in the Azure Monitor. Here is an example of how you can query the pods not in running state in specific namespaces. 

KubePodInventory 
| where Namespace in ("dv","test","prod") 
| where ContainerStatus != "Running" 
| where ContainerStatusReason !in ("", "Completed") 
| distinct Namespace, Name
Run Time range : Last 24 hours E] Save v 14 Share v 1 2 3 4 5 KubePodInventory I where Namespaceän ('dv', 'test', 'prod') I where ContainerStatusu != "Running" I where ContainerStatusReasone ! in ('I", "Completed") I distinct Namespace, Name Results Chart @ Display time (UTC+OO:OO) v Columns v Completed. Showing results from the last 24 hours. > Namespace dv prod Y Name Y app2 app2

The following query includes the name of the AKS cluster and renders the output as a stacked bar chart.

KubePodInventory 
| where Namespace in ("dv","test","prod") 
| where ContainerStatus != "Running" 
| where ContainerStatusReason !in ("", "Completed") 
| distinct ClusterName, Namespace, Name 
| summarize dcount(Name) by ClusterName, Namespace 
| render columnchart kind=stacked100
Run Time range : Last 24 hours E] Save v 14 Share v + New alert rule Export v Pin to dashboard 1 2 3 4 5 6 7 8 KubePodInventory I where Namespace in ('dv', 'test', 'prod') I where ContainerStatus ! = "Running" I where ContainerStatusReason ! in ('I", "Completed") I distinct ClusterName, Namespace, Name summarize dcount(Name) by ClusterName, Namespace render columnchart kind=stacked100 Results Chart @ Display time (UTC+OO:OO) v Completed. Showing results from the last 24 hours. 00:00.7 2 records z o 100 50 dv O aksdemol ClusterName prod Activate Windows Go to Settings to activate Windows.

You can include multiple AKS clusters in the scope in which this query is executed by clicking on [Select scope] hyperlink.


Create an Azure Dashboard panel with this output by clicking on [Pin to dashboard] button.

AIG Demo v/ Private dashboard New dashboard v Auto refresh : Off Analytics rq-aksdashboard CD Refresh Full screen UTC Time : Past 24 hours aksdemol test 62 Edit Share Add filter ClusterName Download aksdem02 Clone e Assign z o 100 75 50 25 dv prod

You can also execute this Kusto query directly using powershell.

$workspaceName = "DefaultWorkspace-6637b095-xxxx-xxxx-xxxx-xxxxxxxxxxx-EUS" 
$workspaceRG = "defaultresourcegroup-eus" 
$WorkspaceID = (Get-AzOperationalInsightsWorkspace -Name $workspaceName -ResourceGroupName $workspaceRG).CustomerID 

$query = 'KubePodInventory | where Namespace in ("dv","test","prod") | where ContainerStatus != "Running" | where ContainerStatusReason !in ("", "Completed") | distinct ClusterName, Namespace, Name | summarize dcount(Name) by ClusterName, Namespace' 

$result = Invoke-AzOperationalInsightsQuery -WorkspaceId $WorkspaceID -Query $query -Timespan (New-TimeSpan -days 1) 
$result.results

This allows you to include the results of your custom Kusto queries in any reports you might run using Azure Automation Runbooks.

PowerShell v Q) PS /home/ash> PS /home/ash> PS /home/ash> Q o PS /home/ash> $workspaceName "DefaultWorkspace-6637b095 PS / home/ash> $workspaceRG "defaultresourcegroup-eus" PS / home/ash> $WorkspaceID (Get-AzOperationa11nsightsWorkspace -Name $workspaceName -ResourceGroupName $workspaceRG) . CustomerID 'KubePodInventory I where Namespace in ("dv" , "test" , "prod") I where ContainerStatus ! = "Running" I where Container PS / home/ash> $query I distinct ClusterName, Namespace, Name I summarize dcount(Name) by ClusterName, Namespace' PS / home/ash> $result Invoke-AzOperationa11nsightsQuery -Workspaceld $WorkspaceID -Query $query -Timespan (New-TimeSpan -days 3) PS / home/ash> $result.results ClusterName Namespace dcount_Name aksdemol aksdemol aksdem02 dv prod test 1 1 1 PS / home/ash> PS / home/ash> PS /home/ash>