Build Your First AI App, Part 1: How AI Apps Actually Work (Tokens, Prompts, APIs and Your First LLM Call in Python)

Many students want to "build something with AI" for their portfolio and get stuck in the same place: they can use ChatGPT, but have no idea what happens between the text box and the answer. This series fixes that. Over five parts you will build StudyBuddy, an AI study assistant for your college notes, and finish with a deployed app you can show to recruiters. Part 1 is the foundation. We will open up an AI app, look at every moving part, set up your machine properly, and make your first real call to a large language model (LLM) in about fifteen lines of Python.

What you will build in this part

• A clear mental model of how any AI app works, from the chat box to the model and back
• A clean Python project with a virtual environment and a protected .env file
• A 15 line script that sends a question to an LLM and prints the answer
• A small command line chat that streams replies and remembers the conversation

What you need

• A laptop with Python 3.10 or newer (Windows, macOS or Linux all work)
• A code editor such as VS Code
• Either a free API key from a hosted provider, or Ollama installed if your machine can run a model locally
• Basic Python: variables, lists, dictionaries, functions and loops

Meet StudyBuddy and the series roadmap

StudyBuddy is the project we will grow across the series. By the end it will let a student upload their semester notes as PDFs, ask questions about them, get answers grounded in those notes, generate quiz questions, and use a few simple tools. It is small enough to finish in a few weekends and real enough to discuss in an interview.

Part	Topic	What StudyBuddy gains
1	How AI apps actually work	A working LLM call and a terminal chat
2	Build a chat app with Python and Streamlit	A proper web interface with chat history
3	Add your own data with RAG	Answers based on your own PDF notes
4	Tools, function calling and agents	Quiz generation, a calculator, and other actions
5	Deploy, secure and showcase	A public URL, secrets handling, and a portfolio write up

The goal is not to copy code. The goal is to understand each layer well enough that you could rebuild StudyBuddy for a completely different use case on your own.

The anatomy of an AI app

Almost every AI product, from a coding assistant to a support bot, is built from the same five layers.

1. User interface (UI). The chat box, buttons and file upload the user sees. In this series we use the terminal first, then Streamlit in Part 2.
2. Backend. Your Python code. It receives input, decides what to do, calls services and returns the result. For small apps the UI and backend live in the same Python file.
3. Prompt layer. The logic that turns a raw user question into a well structured request: a system instruction, the conversation so far, any retrieved notes, and the new question. This is where most of your app's "intelligence" is designed.
4. LLM API. The model itself, reached over HTTPS. You send text, it sends text back.
5. Optional data and tools. A document store for your own content (Part 3) and functions the model can ask you to run, such as a calculator or a search (Part 4).

The model is just one component. The value of your app comes from everything around it: the prompt you design, the data you feed in, and the experience you wrap it in.

The request lifecycle, step by step

Here is what happens when a student types "Explain deadlocks simply" into StudyBuddy.

1. Input. The UI captures the text and passes it to the backend.
2. Context gathering. The backend loads the history and, from Part 3, searches the student's notes.
3. Prompt assembly. The prompt layer builds a list of messages: the system instruction, previous turns, retrieved notes and the new question.
4. API call. The backend sends that list to the provider's endpoint, along with the model name and settings such as temperature, authenticated with your API key.
5. Inference. The provider's servers run the model, which generates the reply one token at a time.
6. Response. The reply comes back either all at once or as a stream of small chunks.
7. Post processing. Your code may log usage, handle errors, or run a tool the model asked for.
8. Display and save. The UI shows the answer, and the backend appends it to the history so the next question has context.

Once you see this loop, debugging gets easier. A bad answer usually comes from step 2 or 3 (wrong or missing context), not a "bad model". Print the full messages list before each call when something looks off.

Tokens, context windows and messages

Tokens

LLMs do not read characters or words. They read tokens, which are chunks of text. A common English word may be one token, while rare words, code and Indian language scripts often take several. As a rough rule of thumb, a token is a bit less than one English word. Tokens matter because providers measure usage, limits and cost in tokens.

Context windows

The context window is the maximum number of tokens a model can consider in one request, counting both what you send and what it generates. Everything the model "knows" about your conversation must fit inside it. Windows are large today but never infinite, and longer prompts are slower and cost more. This is why we will trim old messages, and why Part 3 sends only the most relevant chunks of your notes instead of an entire textbook.

Messages and roles

Chat models take a list of messages, and each message has a role:

• system: standing instructions that shape behaviour for the whole conversation. "You are StudyBuddy, a patient tutor. Answer only about academics."
• user: what the human typed.
• assistant: what the model replied earlier. You send these back so the model can see its own previous answers.

messages = [
    {"role": "system", "content": "You are StudyBuddy, a patient tutor for college students."},
    {"role": "user", "content": "What is a deadlock?"},
    {"role": "assistant", "content": "A deadlock is when two or more processes wait on each other forever..."},
    {"role": "user", "content": "Give me a real life example."},
]

The last question only makes sense because the earlier turns are included. Remember that; it matters shortly.

Temperature, streaming and memory

Temperature

Temperature controls randomness. Low values (close to 0) make the model pick the most likely next tokens, which gives focused, repeatable answers. Higher values make output more varied and creative, but also more likely to drift. For a study assistant explaining syllabus concepts, a low value such as 0.2 to 0.4 works well. Go higher for brainstorming. Accepted ranges differ by provider, so check their docs.

Streaming

Without streaming, your app waits until the whole reply is generated and then shows it at once. With streaming, the provider sends tokens as they are produced, and you print them immediately. The total time is similar, but the user sees the first words within a moment instead of staring at a blank screen.

Statelessness and conversation memory

The gotcha that confuses most beginners: the LLM API is stateless. It does not remember your previous request. Each call is a fresh start. When ChatGPT appears to "remember" what you said earlier, the application is quietly resending the earlier messages with every new request.

So memory is your job. The simplest approach is to keep a Python list of messages, append each user question and each assistant answer, and send the whole list every time. Because of the context window, you also trim old turns once the list grows.

The model has no memory. Your app has memory. Whatever you do not send in this request does not exist for the model.

Latency and cost basics

Two numbers decide whether an AI app feels good and stays affordable.

Latency is how long the user waits. It depends on model size, provider load, your network, and how many tokens go in and come out. Generated tokens are usually the slow part, so "5 short bullet points" returns faster than "explain in detail". Smaller models are generally quicker, and streaming improves how fast the app feels.

Cost on paid tiers is usually calculated per token, often with different rates for input and output tokens. Free tiers typically come with usage limits rather than charges. Numbers change often, so read your provider's pricing and limits page.

Practical habits that keep both under control:

• Trim conversation history instead of sending everything forever
• Send only relevant notes, not whole documents (Part 3)
• Ask for concise answers when that is all you need
• Print response.usage while developing so you can see token counts

Choosing a provider: hosted API or local model

You have two broad options, and both work with the exact same code in this series.

	Hosted API with a free tier	Local model with Ollama
Setup	Sign up, create a key	Install Ollama, download a model
Hardware	Any laptop with internet	Needs decent RAM, a GPU helps a lot
Quality	Access to large, capable models	Smaller open models, quality varies
Speed	Usually fast	Depends entirely on your machine
Privacy	Data goes to the provider	Data stays on your laptop
Limits	Rate limits on free plans	No external limits, only your hardware

Our recommendation: use a hosted free tier for learning and for the deployed version, and try Ollama alongside it if your laptop can handle it.

OpenAI compatible endpoints

This is the trick that makes the whole series provider agnostic. The official openai Python SDK lets you change the base_url it talks to. Many providers, including Google Gemini, Groq and a local Ollama server, offer OpenAI compatible endpoints, which means they accept the same request format. So you write your code once with the openai library and switch providers by changing two or three environment variables.

Provider	Typical base URL (verify in their docs)
Google Gemini	https://generativelanguage.googleapis.com/v1beta/openai/
Groq	https://api.groq.com/openai/v1
Ollama (local)	http://localhost:11434/v1

Model names change often and differ by provider, so we keep the model in a single MODEL variable. Open your provider's model list, pick a current model that is available on your plan, and paste its exact name. For Ollama, use the name of a model you have pulled, which you can see with ollama list.

Getting and protecting your API key

In your provider's developer console, open the API keys section and generate a key. Copy it once; some consoles never show it again.

Treat that key like a password:

• Never paste it directly into your code
• Never commit it to GitHub. Bots scan public repositories for leaked keys constantly
• Never share it in screenshots, chats or reports
• If a key leaks, delete it in the console immediately and create a new one

For Ollama, there is no real key. The SDK still requires a value, so any placeholder string such as ollama works.

Set up your environment the right way

Create a project folder and a virtual environment, which keeps this project's packages separate from the rest of your system.

mkdir studybuddy
cd studybuddy

# Create the virtual environment
python -m venv .venv

# Activate it (macOS / Linux)
source .venv/bin/activate

# Activate it (Windows PowerShell)
.venv\Scripts\Activate.ps1

# Install the two packages we need for Part 1
pip install openai python-dotenv

Your prompt shows (.venv) when active; reactivate it in every new terminal.

Now create a file named .env in the project folder. This holds your secrets and settings as environment variables:

# .env  (example for a hosted provider)
LLM_BASE_URL=https://api.groq.com/openai/v1
LLM_API_KEY=paste-your-real-key-here
LLM_MODEL=your-model-name

# For Ollama instead, use:
# LLM_BASE_URL=http://localhost:11434/v1
# LLM_API_KEY=ollama
# LLM_MODEL=your-pulled-model-name

Then immediately create a .gitignore so the key and the environment folder never reach GitHub:

.env
.venv/
__pycache__/

Create .gitignore before your first commit, not after. Once a key is in your Git history, deleting the file does not remove it.

Your first 15 line script that calls an LLM

Create hello_llm.py:

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()  # copies values from .env into environment variables

MODEL = os.environ.get("LLM_MODEL", "your-model-name")
client = OpenAI(base_url=os.environ["LLM_BASE_URL"], api_key=os.environ["LLM_API_KEY"])

messages = [
    {"role": "system", "content": "You are StudyBuddy, a patient tutor for Indian college students. Use simple English."},
    {"role": "user", "content": "Explain normalization in DBMS in 5 short bullet points."},
]
response = client.chat.completions.create(model=MODEL, messages=messages, temperature=0.3)
print(response.choices[0].message.content)
print("Tokens used:", response.usage.total_tokens if response.usage else "not reported")

Run it with python hello_llm.py. Map each line back to the lifecycle: load_dotenv and os.environ handle secrets, the OpenAI client points at your provider, messages is the prompt layer, create is the API call, and print is your very simple UI. Change the system prompt to "Answer like a strict professor" and run it again.

Level up: a streaming chat with memory

Now combine streaming and memory into a real conversation. Create chat_cli.py:

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
MODEL = os.environ.get("LLM_MODEL", "your-model-name")
client = OpenAI(base_url=os.environ["LLM_BASE_URL"], api_key=os.environ["LLM_API_KEY"])

SYSTEM_PROMPT = "You are StudyBuddy, a patient tutor for college students. Keep answers short and clear."
MAX_TURNS = 10  # keep only the last 10 user/assistant pairs


def trim_history(history, max_turns=MAX_TURNS):
    # Keep the system message plus recent turns to stay inside the context window
    system, rest = history[:1], history[1:]
    return system + rest[-max_turns * 2:]


def main():
    history = [{"role": "system", "content": SYSTEM_PROMPT}]
    print("StudyBuddy is ready. Type 'quit' to exit.")
    while True:
        question = input("\nYou: ").strip()
        if question.lower() in {"quit", "exit"}:
            break
        history.append({"role": "user", "content": question})
        history = trim_history(history)

        stream = client.chat.completions.create(
            model=MODEL, messages=history, temperature=0.4, stream=True
        )
        print("StudyBuddy: ", end="", flush=True)
        answer = ""
        for chunk in stream:
            if chunk.choices and chunk.choices[0].delta.content:
                piece = chunk.choices[0].delta.content
                answer += piece
                print(piece, end="", flush=True)
        print()
        history.append({"role": "assistant", "content": answer})


if __name__ == "__main__":
    main()

Ask "What is a deadlock?", then "Give me an example". The second answer works only because history carries the first exchange. Try commenting out the final history.append(...) line and ask the same pair again: the model loses track instantly. The if chunk.choices check matters too, because some providers send a final chunk with no choices, and skipping that check causes a confusing IndexError.

When things go wrong

Symptom	Likely cause	Fix
KeyError: 'LLM_API_KEY'	.env not found or variable misspelled	Run the script from the project folder and check spelling
401 or authentication error	Wrong, expired or deleted key	Generate a new key and update .env
404 or "model not found"	Model name is outdated or unavailable	Copy a current name from the provider's model list
429 or rate limit error	Too many requests on the free tier	Wait, slow down, and check your plan's limits
Connection refused	Ollama is not running, or wrong base URL	Start Ollama and confirm the URL ends with /v1

Recap and next part

You learned that an AI app is a stack of layers with one repeatable request lifecycle. You now understand tokens, context windows, roles, temperature and streaming, and the crucial fact that the model is stateless, so your app carries the memory. You protected your key with .env and .gitignore, and wrote a 15 line script plus a streaming terminal chat.

Commit your project to GitHub now, after confirming .env is not included.

Next up, Part 2: Build a chat app with Python and Streamlit. We will turn chat_cli.py into a polished web app using st.chat_message, st.chat_input, st.session_state for memory and st.write_stream for streaming, so StudyBuddy finally looks like a product. Keep an eye on the blog for the next part.

If you want fundamentals like Python, APIs and databases explained in order, our courses cover them step by step, and a personalised career roadmap shows where AI app development fits in your plan.