Projects on Ash Tewari

Who needs Surface Ultra RTX?

Thu, 08 Oct 2026 01:38:44 +0000

Microsoft announced the new Surface Ultra RTX Laptop today. Of course I want the RTX Spark Machine! Who doesn’t? It is awesome! I am being nostalgic about my original Microsoft Surface Pro from 2013 though. I have fond memories of developing and testing Windows 8 Apps and Windows Phone Apps(remember those?) on it.

The spec sheet of my Surface Pro 1 from 2013 is ancient: 3rd-gen Intel Core i5, 4 GB of RAM, 128 GB SSD. Windows support ended years ago.

I decided to wipe it and give it a new life with xubuntu.

xubuntu 26.04 refused to boot at first. The culprit turned out to be the installation media rather than Linux: the Surface was hitting USB enumeration and read errors with my original flash drive. Safe graphics made no difference. A different stick, USB 2.0 this time, dropped me straight into the live environment. Live environment was extremely choppy and slow. I was able to complete some basic validations. I settled on xubuntu 24.04 LTS for the permanent install, and I will revisit 26.04 now that the basic setup works.

Then the machine turned out to be more usable than I expected. Wi-Fi, Type Cover, and the Touch Screen all worked, and Firefox, Git, Python, terminals, SSH and lightweight dev tooling all run fine. XFCE and Ubuntu’s package ecosystem give me a current Linux environment that still leaves room to work in 4 GB of RAM. The Surface Pro’s compact form factor makes it light and portable next to my Lenovo Legion behemoth. The whole Surface Pro is probably lighter than the Legion’s power brick.

I am not going to run local LLMs, a Kubernetes cluster or a dozen containers on it. As a small Linux terminal, SSH client, coding machine, browser or home-lab console, it has plenty of life left.

I will be using it as a host for coding agents. The model runs in another local headless server (more on that in another blog post) or in the cloud, so what is left on this machine is an orchestration loop, HTTP calls, a git checkout and a shell, over tailscale. That is an I/O problem, and a 2013 i5 should do just fine. Headless over SSH, 4 GB should be enough. I’ll find out. The other appeal is blast radius. An agent that runs shell commands gets a cheap, disposable machine of its own instead of my main one.

There is something satisfying about taking hardware at the end of its commercial life and saving it from landfill.

Agentic finetuning using a custom harness

Fri, 25 Sep 2026 02:56:11 +0000

The goal was to figure out whether a small model running on my own hardware could do the job of intent classification and structured parameter extraction well enough for one of my use-cases - read a request like “add 8 spare tent stakes to the camping box” and turn it into something the application can act on.

I created an agentic fine-tuning and evaluation loop with a custom evaluation harness: an AI agent proposes one change at a time, a custom evaluation harness measures it against a frozen test set, and every run gets appended to a log with its settings, result and next decision. The agent does the all the work of dataset generation, training, evaluation and diagnosis.

Setup

I built a fixed test set of 45 requests covering nine intents, five cases each, and froze it. Every model, change and training run was measured against those same 45 cases. The output metrics of each run were: intent accuracy, meaning did it pick the right action, and parameter accuracy, meaning did it extract the right details. A third important number is: fallbacks, the count of cases where the model declined to answer and returned “unclear.”

The OpenAI baseline (gpt-5.6-luna) was created and frozen. It scored 93.33% intent and 71.11% parameter accuracy with no fallbacks.

All local training ran on a consumer RTX 3060, in WSL2 with CUDA. LoRA fine-tuning, rank 16, alpha 32, learning rate 1e-4, batch size 4. A 15-epoch run takes about 20 minutes.

The entire source, including training data, eval harness, scripts and agent skill is on GitHub - needle-tuner.

Part 1 - needle2

The first local comparison with needle2 model from Cactus. It scored 4.44% intent accuracy. The problem turned out to be how I was asking the question. I had given the model one abstract, generic “classify this” tool. I replaced it with eight concrete tools, one for each real action the application supports.

The result went to 62.22% intent and 68.89% parameter accuracy.

With this reasonable starting point, I fine-tuned on a small purpose-built training set and ran a structured loop: change one thing, measure against the frozen test set, write down the result.

Needle2 configuration	Intent	Parameter
Base model, redesigned tools, no training	62.22%	68.89%
Best fine-tuned run (5 epochs, rank 8)	64.44%	62.22%
Longer training (30 epochs)	57.78%	64.44%
Heavier defaults	42.22%	51.11%

The best training result gained two points on intent accuracy over no training at all, and lost six on parameters. More training made things worse. The heaviest run also refused to answer far more often, with 15 fallbacks against 9 for the untrained model. The results were noisy enough to make any single comparison suspect. Two runs of the identical best configuration measured 64.44% and 60.00%, so I stopped treating one good run as evidence of anything. Validation loss kept improving while held-out accuracy got worse. Had I watched the training dashboard instead of the frozen test set, I would have shipped a regression. I stopped at this point.

Part 2 - needle3

A new version of the model family had shipped in the mean time, with a different architecture and a different native engine. Untrained, needle3 measured 71.11% intent and 66.67% parameter accuracy with 7 fallbacks. I modified the agentic loop to support the new model and executed again.

A one-epoch smoke test scored 68.89% and 73.33%, which is one run on one epoch and proves nothing on its own. A full 30-epoch run at the wrapper defaults scored 71.11% and 66.67%, identical to the untrained base. However, the per-intent breakdown had shifted underneath, some intents up and some down, while the totals stayed flat. Validation loss bottomed out at epoch 13 and climbed for the remaining 17 epochs while training loss kept falling. Same overfitting pattern needle2 had shown.

The issue

Instead of tuning another hyperparameter, I stopped reading aggregate scores and started reading individual failures.

The training data and the held-out test set agreed on which intent was correct. They disagreed on the convention for writing the parameters down.

The first problem was descriptive adjectives. For delete requests, the test set strips condition words from item names, so “the broken toaster” is expected to yield ’toaster’. All 12 authored delete rows in my training data did the opposite, keeping ‘cracked mixing bowl’ and ’expired pain reliever’ intact.

The second problem was a wrong key name. For inventory-viewing requests, the training data used a ’location’ key for the container slot while every other intent, and the test set, used ‘boxLabel’. It also carried a ‘scope’ key that appears in zero held-out cases.

The model was learning my convention faithfully and being marked wrong for it, on 100% of those cases. That also explains why longer training hurt: more epochs meant learning the wrong answer more thoroughly.

The Fix

I corrected 12 delete rows and 20 view-inventory rows, regenerated the derived datasets, re-pinned the integrity hashes and re-ran the validation suite. No hyperparameter change and dropped Epoch count from 30 to 15, taken from the previous run’s validation-loss minimum.

Result: 82.22% intent and 75.56% parameter accuracy, 6 fallbacks.

Run	Intent	Parameter	Fallbacks
Needle2 base	62.22%	68.89%	9
Needle2 best fine-tune	64.44%	62.22%	not recorded
Needle3 base	71.11%	66.67%	7
Needle3 fine-tuned, original data, 30 epochs	71.11%	66.67%	6
Needle3 fine-tuned, corrected data, 15 epochs	82.22%	75.56%	6
OpenAI (frozen baseline)	93.33%	71.11%	0

That is 11 points of intent accuracy over its own base model and 20 over needle2’s base. It is also the first time a local model beat the hosted API on parameter accuracy, 75.56% against 71.11%. Intent accuracy is still 11 points behind.

The per-intent movement matches the diagnosis. Delete went from 80%/40% to 100%/80%. Search went from 60%/60% to 100%/100%. Update improved as well, since the bare-noun convention is shared by every intent that names an item.

Validation loss behaved differently too. With the corrected data it declined every one of the 15 epochs, with none of the mid-run turnaround the unfixed run showed. The contradictory labels had been generating the overfitting signal themselves.

Parameter extraction already beats the API. Intent classification is 11 points behind, and most of the remaining gap sits in one diagnosed failure mode: deciding whether a vague request is actionable.

The next lever is contrastive training examples sitting right on that boundary. After that the options get expensive: changes to how the model produces its answers, training that explicitly teaches it what a wrong answer looks like, or a different architecture.

Hyperparameter tuning and the size of the training dataset should also be revisited to explore further improvements to intent classification.

Because the same 45 cases were used to decide which experiment to run next, they are now a model-selection set rather than an unbiased final exam. Before anyone calls this production-ready it has to be tested against a separate set of cases it has never influenced.

The project also produced a reproducible pipeline along the way. A frozen test set, hash-pinned datasets, a documented training process, a log of every attempt including the failures, and a default path that validates without training and without calling paid APIs. That infrastructure is the reason the annotation issue was discoverable at all.

Giving OpenClaw Secure Access to Cloud Services Without Sharing Your Password

Mon, 06 Apr 2026 04:48:17 +0000

Device Code Flow has been built into OAuth2 for years, originally designed for TVs and game consoles. It works just as well for a Docker container. It requires no credentials stored on the agent machine. It gives you narrow, revocable access that the agent cannot exceed.

I have OpenClaw deployed in a sandboxed docker container on a dedicated host. I communicate with it via a Telegram bot, in a locked-down chat.

I wired the device-code authentication pattern up with OpenClaw and Microsoft (Personal/Consumer) services, but the authentication approach works with any app and any command-line tool you want to run in a headless environment, including Custom APIs. Device code flow + Entra ID app registrations give you a zero-stored-credentials gateway to any HTTP API you can build, not just Microsoft Graph.

What Is Device Code Flow?

OAuth2 device code flow (RFC 8628) was designed for “input-constrained devices” — things without a keyboard or a browser. Think of how you sign in to Netflix on a smart TV: a short code appears on screen, you visit a URL on your phone, you type the code in, and the TV logs in. You never type your password on the TV.

The official name for that pattern is the device authorization grant. A Docker container is, from the protocol’s perspective, exactly the same kind of device. It has no browser. It cannot perform an interactive redirect. But it can make HTTP requests, and that is all it needs.

The flow has three steps:

1. The app posts to the identity provider’s device code endpoint. It gets back a short user code (like “WDJB-MJHT”), a verification URL, and a polling device code.

2. The user visits the URL on any browser, on any device — their phone, laptop, anything — and enters the short code. They sign in normally, with their usual credentials and MFA if it is enabled.

3. The app polls the token endpoint in the background. Once the user finishes signing in, the poll returns a real access token and a refresh token. The app stores these and uses them for API calls going forward.

The app never sees the password. The password goes directly from the user’s browser to the identity provider. The app only ever handles two things: a short temporary code that it sends to the user, and a token that the identity provider gives back once the user has authenticated.

OpenClaw running in a Docker container fits this model exactly.

Which Services Support this?

Device code flow is not a Microsoft-only feature. It is a standard OAuth2 extension (RFC 8628) and most major identity providers support it — including Microsoft, Google, GitHub, and AWS. If a service uses Okta or Auth0 for identity, those support it too.

The Security Architecture

The agent never sees your credentials. Your password is typed in your browser, on your device, to your identity provider’s servers. It never touches the container. The agent only handles a short temporary code to give you, and a token issued by the provider once you have authenticated.

The device code is one-time and short-lived. After the user authenticates, the code is permanently invalidated. An intercepted code is useless without the user’s credentials and MFA.

Scopes define the ceiling. You configure the OAuth app or app registration to request only specific permissions. The agent cannot exceed those scopes. If you configure read-only access to email, the token cannot be used to send or delete email.

This is the authentication pattern that many of the consumer devices you already own use to access your accounts on your behalf. Your TV’s YouTube app, your smart home hub’s Google integration, the GitHub CLI you use on your workstation — these all use device flow. You have been trusting it for years without knowing what it was called.

How the implementation works

There are three components, and they run as separate Docker services that talk to each other through the Docker socket.

The app sidecar is your application container. It runs your CLI tool and your auth logic. It does nothing on its own. Its only job is to hold the authenticated token cache and execute commands on demand when OpenClaw calls into it.

OpenClaw is the AI agent container. It connects to Telegram, understands natural language, and knows which skills to run for which requests. It does not know anything about OAuth or your specific app. It just runs shell commands you specified and reports results back to you.

The Telegram auth bridge is a small script that lives inside the sidecar. It registers a device code callback, requests an auth flow, and uses the Telegram Bot API to forward the code to you — then confirms when authentication completes.

The compose file looks roughly like this:

services:
 myapp-sidecar:
 image: ghcr.io/youruser/myapp:latest
 command: ["sleep", "infinity"]
 environment:
 - XDG_DATA_HOME=/data
 - TELEGRAM_BOT_TOKEN=${TELEGRAM_BOT_TOKEN}
 - TELEGRAM_CHAT_ID=${TELEGRAM_CHAT_ID}
 volumes:
 - ${APP_DATA_DIR:-./app-data}:/data
 restart: unless-stopped

 openclaw-gateway:
 build:
 context: .
 dockerfile: Dockerfile.openclaw
 volumes:
 - /var/run/docker.sock:/var/run/docker.sock
 - ./skills:/app/skills
 group_add:
 - "${DOCKER_GID:-999}"
 environment:
 - TELEGRAM_BOT_TOKEN=${TELEGRAM_BOT_TOKEN}
 ports:
 - "18789:18789"
 restart: unless-stopped

The Callback Pattern

The auth manager has a method called set_device_code_callback. You pass it a function, and when the device code is ready, the auth manager calls your function with the code and the verification URL rather than trying to open a browser.

class AuthManager:
 def __init__(self, client_id: str, authority: str) -> None:
 self._client_id = client_id
 self._authority = authority.rstrip("/")
 self._on_device_code = None
 self._tokens = self._load_cache()

 def set_device_code_callback(self, fn) -> None:
 self._on_device_code = fn

 async def _device_code_auth(self) -> None:
 async with httpx.AsyncClient() as client:
 resp = await client.post(
 f"{self._authority}/oauth2/v2.0/devicecode",
 data={"client_id": self._client_id, "scope": OAUTH_SCOPES},
 timeout=30,
 )
 flow = resp.json()
 user_code = flow["user_code"]
 verification_uri = flow["verification_uri"]
 device_code = flow["device_code"]
 interval = flow.get("interval", 5)
 expires_in = flow.get("expires_in", 300)

 if self._on_device_code:
 self._on_device_code({
 "user_code": user_code,
 "verification_uri": verification_uri,
 "message": flow.get("message", ""),
 })
 else:
 try:
 webbrowser.open(verification_uri)
 except Exception:
 pass

 deadline = time.time() + expires_in
 while time.time() < deadline:
 await asyncio.sleep(interval)
 async with httpx.AsyncClient() as client:
 resp = await client.post(
 f"{self._authority}/oauth2/v2.0/token",
 data={
 "client_id": self._client_id,
 "grant_type": "urn:ietf:params:oauth:grant-type:device_code",
 "device_code": device_code,
 },
 timeout=30,
 )
 body = resp.json()
 if resp.status_code == 200 and "access_token" in body:
 self._store_tokens(body)
 return
 error = body.get("error", "")
 if error == "authorization_pending":
 continue
 elif error == "slow_down":
 interval += 5
 elif error in ("authorization_declined", "expired_token"):
 raise AuthError(error)
 else:
 raise AuthError(body.get("error_description", "Auth failed"))

The polling loop handles the three error codes the spec defines: authorization_pending (keep waiting), slow_down (back off and increase the interval by 5 seconds), and the terminal errors that stop polling.

The Telegram Bridge

With the callback mechanism in place, the Telegram bridge is just a script that registers a callback and uses the Telegram Bot API to deliver the code to you:

import asyncio, os, sys, traceback
import httpx
from your_app.auth import AuthManager
from your_app.config import Settings

TELEGRAM_BOT_TOKEN = os.environ["TELEGRAM_BOT_TOKEN"]
TELEGRAM_CHAT_ID = os.environ["TELEGRAM_CHAT_ID"]

async def send_telegram(text: str) -> None:
 url = f"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/sendMessage"
 async with httpx.AsyncClient() as client:
 await client.post(url, json={
 "chat_id": TELEGRAM_CHAT_ID,
 "text": text,
 "parse_mode": "Markdown",
 }, timeout=30)

async def main() -> int:
 settings = Settings()
 auth = AuthManager(settings.client_id, settings.authority)

 if auth.is_signed_in():
 await send_telegram("Already authenticated.")
 return 0

 def on_device_code(info: dict) -> None:
 msg = (
 "*Auth Required*\n\n"
 f"Go to: {info['verification_uri']}\n"
 f"Enter code: `{info['user_code']}`\n\n"
 "Complete sign-in in your browser and I'll confirm when done."
 )
 try:
 asyncio.get_event_loop().create_task(send_telegram(msg))
 except RuntimeError:
 import requests
 requests.post(
 f"https://api.telegram.org/bot{TELEGRAM_BOT_TOKEN}/sendMessage",
 json={"chat_id": TELEGRAM_CHAT_ID, "text": msg, "parse_mode": "Markdown"},
 timeout=10,
 )

 auth.set_device_code_callback(on_device_code)
 try:
 user_info = await auth.sign_in()
 await send_telegram(f"*Signed in* as {user_info['name']} ({user_info['email']})")
 return 0
 except Exception as exc:
 await send_telegram(f"*Auth failed:* {exc}\n```{traceback.format_exc()[:800]}```")
 return 2

if __name__ == "__main__":
 sys.exit(asyncio.run(main()))

The OpenClaw Skill

On the OpenClaw side, a “skill” is a markdown file. When OpenClaw sees a trigger phrase in Telegram, it runs the associated shell command. The markdown file looks like this:

---
name: myapp-auth
description: Authenticate with the cloud service via device code flow, coordinated through Telegram.
---

## myapp-auth

Use this skill when the user sends "/auth", "authenticate", or "sign in".

Run the following command and wait up to 360 seconds for it to complete.
The script sends the device code to the user via Telegram and confirms when done.

 docker exec myapp-sidecar-1 python /app/scripts/telegram_auth.py

Summary

Device code flow is a mature, widely-supported OAuth2 pattern that maps naturally onto OpenClaw running in Docker container. With this pattern OpenClaw never handles your credentials. It gets scoped, revocable tokens from the identity provider, after you have authenticated in your own browser. A Telegram bot is all you need to coordinate the handoff of the temporary code.

The approach works for many use-cases across Microsoft accounts, Google, GitHub, AWS, Auth0, Okta and others. It supports exactly the kinds of personal automations that make an OpenClaw genuinely useful in daily life.

References

Understanding RAG @ All Things Open AI 2025

Mon, 31 Mar 2025 02:29:39 +0000

What I learned while creating Bookshelf (open source RAG Application)

I had the opportunity to speak at the AllThingsOpen.ai 2025 Conference recently. It was a wonderful experience to learn from and share with so many amazing fellow presenters and attendees. All Things Open AI 2025 session recordings are available now.

I spoke about several key aspects of Retrieval-Augmented Generation (RAG) and vector databases. The main topics I addressed included The Core Concepts of RAG, Embedding Models vs. Inference Models, Handling Multiple Embeddings and various Optimizations Beyond Naive RAG such as chunking strategies, enrichment with metadata, auto-merging retriever, and reranking models. Finally, I touched on Local Execution and GPU Utilization.

You can watch the recording here: YouTube - How I created Bookshelf: An Open Source AI-Powered Personal Knowledge Base - Ash Tewari

The slides can be downloaded from here: (Google Drive Link)

References and Links:

AI Powered Bookshelf - Blog Post
HuggingFace - MTEB Leaderboard
PyTorch - Local Execution
Medium - A Brief Introduction to Retrieval Augmented Generation(RAG)
[Microsoft - RAG Architecture](http://Design and develop a RAG solution)
Amazon - Retrieval Augmented Generation

SPCHR: Speech To Text App for Windows

Mon, 10 Feb 2025 04:32:25 +0000

I’m excited to announce the release of SPCHR. Pronounced as “speaker”. I know, I know. My apologies, it was too late at night and that’s the best I could come up with ;)

It is a Windows desktop application for speech-to-text transcription.
You can use it to “add” voice input to applications that don’t natively support it. It works anywhere you can type or paste text.
You can use it entirely locally on your PC if you like, so it is private and secure.

Key Features

Flexible: SPCHR can use either Azure Speech Services or a local OpenAI Whisper model, giving you the flexibility to choose between cloud-based and local transcription.
Zero Configuration: The application works immediately with local Whisper processing - no account setup required!
Global Hotkey: Start and stop recording from any application with Ctrl+Alt+L.
Seamless Integration: Transcribed text is automatically pasted into your active window.

Getting Started

Clone the repository
Build the solution using Visual Studio and run it
Press Ctrl+Alt+L and start speaking
Watch as your words appear in your active window
For those wanting cloud-based transcription, simply add your Azure Speech Services credentials to the configuration file.

Open Source

SPCHR is released under the MIT License, and I welcome contributions from the community. Whether you’re interested in adding features, fixing bugs, or improving documentation, your help is appreciated.

Check out GitHub repository

What’s next

This is just the beginning for SPCHR. Several improvements can be done:

Additional language support
Customizable hotkeys
Installer
UI Enhancements
AI Enhanced Features

Try it out

Ready to transform how you interact with your computer? Visit the GitHub repository to get started with SPCHR today.

I’m excited to see how you’ll use SPCHR in your workflow and look forward to your feedback!

WhichBox - AI Assistant App

Sun, 04 Aug 2024 17:37:56 +0000

Do you remember which one of those boxes in the garage has your old phone? Or that toy your child wants to play with again? Or the tripod you need for your weekend trip? Oh what about that massager that you could really use right now! You would probably have to dig through those boxes to find it. Same problem when moving. There are always some boxes that you don’t want to open right away, but you wish you could open just the right one when you need something.

I have created WhichBox- your AI Assistant to find that box. It uses latest Vision AI models to help you find things quickly. Here’s what you do -

Take pictures of the content inside each labelled box to create an inventory.
Use WhichBox to easily identify the box containing the item you are looking for.

You can check it out here - https://whichbox.streamlit.app/

The demo has four photos of labelled boxes with some content in them. Note that there can be multiple photos of the same box. You can take photos as you are filling up the box to capture things at the bottom.

You can ask for a specific thing, like “Find Fitbit” or just “fitbit”

You can go for a general category, like “Camera Equipment”

You can get all boxes containing “USB Adapters”

You can look for “Groot” or if you can’t remember the name of a specific toy then you can look for “all toys of movie characters”

For now, you can bring your own API Key and use your own photos to try this out.

WhichBox : https://whichbox.streamlit.app/

AI Powered Bookshelf

Mon, 29 Apr 2024 00:25:55 +0000

Bookshelf is a Generative AI application built as a rudimentary, but fairly capable, RAG implementation written in python. It can use an open source LLM model (running locally or in the cloud) or a GPT model via OpenAI’s API.

The application is created using streamlit.
I used llama-index for orchestrating the loading of documents into the vector database. Only TokenTextSplitter is currently used. It does not optimize for PDF, html and other formats.
ChromaDb is the vector database to store the embedding vectors and metadata of the document nodes.
You can use any open source embeddings model from HuggingFace.
Bookshelf will automatically use the GPU when creating local embeddings, if the GPU is available on your machine.
You can use OpenAI embeddings as well. There is no way to use a specific OpenAI embedding model or configure the parameters yet.
Use OpenAI API or any OpenAI compatible LLM API (using LMStudio, Ollama or text-generation-webui) of your choice.
There is a live demo on streamlit cloud - https://bookshelf.streamlit.app/
The demo allows only OpenAI integration. You can run it locally for accessing Open Source embedding models and LLMs.

Live demo - https://bookshelf.streamlit.app/

You will need your OpenAI api key for the demo.

If you are running it locally, you will have the option of using an Open Source LLM instance via an API Url. In the screenshot, I am using an open source Embedding Model from HuggingFace (sentence-transformers/all-mpnet-base-v2) and The local LLM server at http://localhost:1234/v1

Collections tab shows all collections in the database. It also shows the names of all the files in the selected collection. You can inspect individual chunks for the metadata and text of each chunk. You can delete all contents of the collection ( there is no warning ).

You can modify the collection name to create a new collection. Multiple files can be uploaded at the same time. You can specify if you want to extract metadata from the file contents. Enabling this option can add significant cost because it employs Extractors which use LLM to generate title, summaries, keywords and questions for each document.

On the Retrieve tab, you can query chunks which are semantically related to your query.

On the Prompt tab, you can prompt your LLM. The context as well as the Prompt Template is editable.

Here is an example of using the context retrieved from chunks in the Vector database to query the LLM.

This inference was performed using Phi3 model running locally on LMStudio.

Code is on Github - https://github.com/ashtewari/bookshelf

Have fun!

Prompt Engineering and Security - Custom GPT

Sun, 21 Jan 2024 23:01:24 +0000

I have been delving into Advanced Prompt Engineering and Security techniques for Large Language Models (LLMs). As an exercise, I have created a custom GPT in ChatGPT to help practice spelling of English words. Spell It GPT is secured with Advanced Prompt Engineering techniques to guard against common attack vectors, including Direct Prompt Injection, Prompt Leaking, Role Playing, Simulation, DAN and Code Injection.

Spell-It-GPT

Play with the Spell It GPT and try to break it. It is not impossible but (probably) fairly difficult to do ;) Regardless, practice spelling and have fun!

Switch to Voice Mode in ChatGPT Mobile App to practice spelling and put headsets on for best results!

Spell It: https://bit.ly/3Ua0oib

Here are some resources you can learn more about Prompt Engineering and Security:
Advanced Prompt Engineering Techniques
OWASP Top 10 for Large Language Model Applications
Educational Resources
Adversarial Prompting in LLMs

KHEL Coin on Ethereum

Tue, 10 May 2022 01:41:59 +0000

KhelCoin (KHEL) is an ERC-20 token available on Ethereum blockchain. It is a Smart Contract created with Solidity, OpenZeppelin, Truffle, Ganache, Remix, web3.js and VSCode. It is currently available on Rinkeby test network. It will be launched on the Mainnet in the near future.

You can get KHEL coins from this website - KHEL Coin ICO

Head over to the ICO website. You will be prompted to install Metamask and connect to your account on Rinkeby testnet. Click on “Buy KHEL Coins” button to get your own coins!

Source code is on github - https://github.com/ashtewari/khelcoin

Puzzle Icon Credit: https://www.flaticon.com/free-icons/puzzle

Universal Windows Platform Application on Raspberry Pi

Sat, 23 May 2020 02:57:40 +0000

Universal Windows Platform (UWP) provides a common app platform on every device that runs Windows 10. The core APIs in UWP are the same on all Windows devices – including Desktop PC, Mobile Phone, XBox, Hololens, IOT devices and others.

You can target specific device capabilities of a device family with extension SDKs, but you don’t have to do that if you are only using core APIs. Those core APIs include a very impressive set of UI capabilities. What that means is that you can create a UWP application using C# and XAML which will run on an ARM based processor on Raspberry Pi – because Windows 10 IOT Core runs on that.

Universal Windows Platform (UWP) has come a long way. I decided to dust off my good old Raspberry Pi 2B and take it for a spin last weekend.

The goal was simply to create a Hello World UWP application, deploy it on Windows 10 IOT running on Raspberry Pi (RPi), without using any device specific extension SDKs. Here is the stack -

Universal Windows Platform App (C#/XAML)
Raspberry Pi Model 2B
Windows 10 IOT Core
Syncfusion Essential Studio 18.1.0.42
Visual Studio 2019

Create a simple UWP application in Visual Studio 2019.

Install Windows 10 IOT Core on Raspberry Pi 2B and connect the device to a 50" screen TV.

Select ARM in the target platform drop down and Click on Remote Machine to Run/Debug Application on RPi device.

Select RPi device for deployment.

Deployment of UWP on RPi is complete.

UWP application running on RPi.

I updated the UWP application to include Syncfusion Chart control. Did you know that you can get a Community License for the entire product line (including Blazor, Xamarin and UWP controls) from Syncfusion?

UWP application is now running on RPi, attached to the Visual Studio debugger running on the development machine (Windows Surface).

No device specific extension SDKs were referenced, just the core UWP APIs were used.

Code is on github.

PackRaft with Xamarin and Microsoft Azure

Sun, 20 Dec 2015 03:22:26 +0000

PackRaft is a cross-platform mobile application built with Xamarin. It integrates with Microsoft Azure services, such as Azure Table Storage and Azure App Service. It is created with Xamarin.Forms for iOS and Android (Windows version will be coming soon).

The purpose of creating this sample application was to expand expertise in cross-platform mobile development with Windows Azure integration, specifically with Xamarin.Forms and Azure App Service. With Xamarin.Forms, it is possible to achieve close to 100% code reuse across these platforms for certain type of apps. Azure App Service provide a set of rich cloud backend services for your mobile applications.

PackRaft allows the user to setup a fleet of vehicles of various seating capacities. Groups of riders are assigned to these rides according to the count of riders in each group. You can specify that certain groups be kept together on a single vehicle - and the app tries to honor those requests if possible. I did this via a rudimentary implementation of a bin packing algorithm. This implementation is deployed as an API App in Microsoft Azure.

PackRaft uses Azure Mobile Services SDK to persist Fleet data to Azure Table Storage and to authenticate users with Facebook and Microsoft accounts. Twitter and Google account authentication is also supported by Azure Mobile Services SDK out of the box and those will also be enabled in PackRaft soon.

PackRaft is available on Android, iPhone and iPad.

Creating Android Apps using C# and Xamarin

Sat, 21 Feb 2015 17:30:25 +0000

You can create native Android and iOS applications using C#, with Xamarin. I explored the capabilities of Xamarin tools and the development experience with Xamarin to create an Android version of CensusMapper. CensusMapper displays U.S. Census data on a Map. I created the original application for Windows 8 (WinRT) using C#. I have since converted it to an MVVM (C#) application, HTML/Javascript (WinJS) version and more recently to a Universal Windows App running on Windows Phone 8.1 and Windows 8.1. In that tradition, now it is faithfully ported to run on Android devices, using C# and Xamarin. CensusMapper is available on Google Play Store.

CensusMapper.Android shows population of each U.S. State. User can tap on a location on the map to show the population (as per 2010 census) of the zip code containing the selected location. User can select a location to view the population compared to the containing geography, as a pie-chart.

This experimental project helped me expand my cross-platform mobile application development skillset in the following areas:

Using Xamarin/C# to create an Android app using Xamarin Studio on a PC and a Mac.
Structuring C#/.NET source code in a Portable Class Library to allow reuse on Windows, Windows Phone, Android and iOS platforms.
Converting the functionality of an existing Windows Universal App (Windows Phone/ WinRT) to an Android app.
Using GoogleMaps API (via Google Play Services) to display data retrieved from U.S. Census API.
Using Bing Maps API for reverse geocoding in an Android app.
Creating custom markers using low-level API in Android.Graphics namespace to display information on the map.
Charting with oxyplot cross-platform charting library.
Working with Android SDK, AVD Manager as well as developer documentation and resources on Android and Xamarin websites in the development lifecycle of the app.
Publishing an app on Google Play Store.

I started with Xamarin.Forms.Maps. It was quick and straight forward to create a Xamarin.Forms app. The tricky part was creating the custom markers (the shape of speech bubble) on the map, instead of using the default push-pin markers . I tried creating a CustomRender to create custom markers and built interactivity with the map elements, but it turned out to be too much hassle. All of that customization would be specific to Android platform only and not reusable on the iOS anyway. So, I decided against using Xamarin.Forms and work directly with Google Maps library. Xamarin.Forms also limits me to Windows Phone 8 compatibility in my PCL. I can’t use Windows Phone 8.1 in any PCL profile compatible with Xamarin.Forms. It also requires a minimum of Android 4.0. The PCL profile # 111 for sharing code between Windows 8, Windows Phone 8.1 as well as Android and iOS, is not compatible with Xamarin.Forms at the time of writing this.

Don’t use the Android emulator for testing. Get a real device. I stopped using the emulator while working on CensusMapper because GooglePlayServices (GPS) dependency was making me jump through way too many hoops. Debugging map functionality on the real device was a much more productive experience. Then I discovered Xamarin’s Android Player. It is awesome. By all means, use it! It is a pleasure to work with, especially on a Mac. Don’t be discouraged by the preview/beta status. It blows the Android SDK virtual devices out of the water, in terms of performance and usability.

I tested on Google Nexus 4 (Android 4.2), Samsung Galaxy SIII (Android 4.3) , BLU Dash JR (Android 4.4) and an older Coby Kyros tablet (Android 2.2). Google Play Services update, required for the app to work, could not be installed on BLU device because of the limited storage space. You must root the device in order to install the GPS updates on an SD card. It is advisable to have a root-friendly device. Google Nexus devices are excellent development devices, allowing you root access and the ability to load custom ROMs. However, it is very important for you to get a few different kinds of devices of various form factors and Android versions/API levels for testing your app’s functional and non-functional aspects.

Android device logs and the IDE build logs are the most important tools you have to resolve problems encountered in the Android app development with Xamarin. These should be the first places to look when troubleshooting issue.

Versions used in this project: Xamarin 3.8. Xamarin Studio 5.7 Xamarin Android 4.20.0

Universal Apps

Thu, 13 Nov 2014 02:50:36 +0000

The story of Windows Universal Apps is the result of advancements in three areas:

The platform convergence between Windows 8.1 and Windows Phone 8.1
The (ongoing) unification of Windows and Windows Phone stores.
Shared Projects and other enhancements in Visual Studio 2013 Update 2 to support cross-platform development.

I have written about my experience in converting an existing Windows 8 app, CensusMapper, to a Universal App on the AIS Blog. Read the article here. I have also created another Universal App, LRMS, using Windows App Studio. This app is available in both Windows and Windows Phone Stores.

I gave a presentation at the Raleigh Code Camp this past weekend : Creating Universal Apps for Windows Phone 8.1 and Windows 8.1. Thanks to everyone who attended the presentation. The session slides for this presentation are available here : Download Slides.

Announcing CodeDemo

Mon, 10 Mar 2014 01:58:22 +0000

It is a challenge to use code snippets in presentations, demos and live coding sessions. I have struggled with this and seen many speakers jumping through hoops to assimilate code snippets in their demos. I had to create something to ease this pain.

Today, I am announcing the general availability of CodeDemo. A tool for instructors, presenters and speakers who create instructional demos, record screen-casts or do live presentations. CodeDemo pastes code snippets into your favorite IDE without getting in your way.

I decided not to open-source CodeDemo to challenge myself to an exercise in commercialization. I decided to learn about entrepreneurship, marketing, sales, licensing and other aspects of software commercialization. It has been an MBA’s worth of education so far. This journey is not over yet. I am taking the Lean approach of creating a Minimum Viable Product. The future of CodeDemo, its feature set and the price, will be shaped by the traction it gets in the market, the feedback from the customers as well as patterns of use reported by the users.

Head over to http://codedemo.net to grab a copy for yourself. Let me know if find it useful - I will be very excited to hear from you.

Here is a quick 2 minute introduction to CodeDemo :

[youtube]aFn3boBAWjw[/youtube]

Windigo 1.1 Released

Mon, 05 Dec 2011 13:59:47 +0000

Windigo is a windows phone app to access your diigo account on windows phone 7.

Create, edit and search diigo bookmarks. Slick, to-the-point interface. Browse and search public bookmarks of other diigo users. Tap on the bookmark for details, including tags. Tap on any tag to narrow your search. Trial version is available.

Windigo 1.1 is out on the windows phone marketplace now. Highlights of this update :

- Last SearchScope used is now persistent

- Fixed search when a search term is picked from suggested words

- Problem report now includes system info

- Handle crash on navigating from main page

- Fixed feature tracking for analytics

Last SearchScope used is now persistent
Fixed search when a search term is picked from suggested words
Problem report now includes system info
Handle crash on navigating from main page
Fixed feature tracking for analytics