Course: ChatGPT for Data Science and Data Analysis in Python
Lesson
Topic: Intro to Prompt Engineering — Practical data workflows with Python and AI
Lesson
Prompt engineering is the craft of designing clear, context-rich inputs that guide AI models toward useful outputs. Good prompts combine clarity, appropriate context, and specific instructions so language models return relevant, actionable responses. In practical data work, prompts can describe the desired data transformation, summarize dataset characteristics, or request code snippets for cleaning and analysis. When working with pandas, recordlinkage, or SQLite, a well-formed prompt helps the model produce code that fits the dataset and the analytic goal. Iterative refinement is essential: start with a focused prompt, inspect the result, and then refine prompts to reduce ambiguity and handle edge cases. For reproducible data workflows, pair prompt instructions with deterministic code patterns such as UUID-suffixed temporary table names for SQLite. Prompt engineering also overlaps with software engineering: validate outputs, log actions, and sanitize inputs before executing generated code. In team settings, standardizing prompt templates increases consistency across analyses and reduces error. Finally, treating prompts as testable artifacts — with examples, expected outputs, and failure modes — makes AI-assisted data work safer and more maintainable.
Learning Outcomes
- Define prompt engineering and list its three core components: clarity, context, specificity.
- Apply prompt-engineering principles to common data tasks: summarization, deduplication, and dynamic SQL.
- Implement safe, reproducible code patterns illustrated by pandas, recordlinkage, and sqlite3 examples.
- Design simple evaluation checks and logging for AI-generated code to reduce runtime risks.
Conceptual Explanation
Prompt engineering shapes the textual input sent to a language model so the model's output maps closely to the user's intent. The three pillars — clarity, context, and specificity — reduce ambiguity and constrain the model's search space. In data workflows, prompts often accompany structured inputs (e.g., a DataFrame head, schema description, or example rows) so the model can produce correct and executable code. Best practices include: provide a brief dataset description, give explicit desired output format (table, CSV, JSON), include sample rows or column names, and state constraints (performance, privacy, deterministic identifiers). Complement prompts with programmatic safeguards: validate generated SQL before execution, append UUIDs to ephemeral table names, and log original prompts and sanitized versions.
| Component | Why it matters | Example prompt fragment |
|---|---|---|
| Clarity | Reduces ambiguous instructions | "Return a CSV with columns: id, name, score" |
| Context | Guides the model with background | "DataFrame has columns: date, state, sales" |
| Specificity | Directs output format and constraints | "Use pandas; limit results to top 10 states by sales" |
Bilingual (EN → বাংলা)
EN: Prompt engineering = designing clear, contextual inputs for AI models.
বাংলা: প্রম্পট ইঞ্জিনিয়ারিং = AI মডেলের জন্য স্পষ্ট ও প্রাসঙ্গিক ইনপুট ডিজাইন করা।
- EN: Clarity — avoid ambiguity. / বাংলা: স্পষ্টতা — দ্ব্যর্থহীন রাখুন।
- EN: Context — provide schema or examples. / বাংলা: প্রসঙ্গ — স্কিমা বা উদাহরণ দিন।
- EN: Specificity — state desired format and constraints. / বাংলা: নির্দিষ্টতা — কাঙ্ক্ষিত আউটপুট ও সীমাবদ্ধতা বলুন।
Important Notes
- Never execute unvalidated SQL or system commands returned by a model without sanitization and manual or automated checks.
- When creating temporary tables in SQLite, append a short UUID to the name to prevent collisions and accidental overwrites.
- For record linkage and fuzzy matching, choose sensible thresholds and test on labelled examples to calibrate precision/recall trade-offs.
- Log prompts, responses, and the final executed code to make AI-assisted analyses auditable and reproducible.
Exercises — True / False / Fill in the Blanks / MCQ / Matching / Descriptive / Creative
True / False
- Prompts should always be as short as possible, regardless of context.
Answer
False. While concision is valuable, omitting necessary context or specificity can make outputs ambiguous. Balance brevity with the context needed for correct results. - Appending a UUID to a temporary SQLite table name helps avoid conflicts during concurrent runs.
Answer
True. Using a deterministic short UUID (or random suffix) prevents name collisions and aids traceability.
Fill in the Blanks
- Three key components of an effective prompt are: __________, __________, and __________.
Answer
Clarity, Context, Specificity. - When performing record linkage, blocking on a field like 'state' is used to __________ the number of comparisons.
Answer
reduce
Multiple Choice (MCQ)
- Which practice reduces the risk of executing malicious SQL returned by a language model?
- Execute immediately to test correctness
- Replace identifiers with sanitized, UUID-suffixed table names and validate SQL text first
- Trust the model if the prompt was clear
Answer
b) Replace identifiers with sanitized, UUID-suffixed table names and validate the SQL before execution. Always validate and sanitize. - Which encoding is best for a low-cardinality categorical feature where order matters?
- One-hot encoding
- Label encoding (ordinal)
- Target encoding
Answer
b) Label encoding (ordinal), because order matters. For nominal features, use one-hot or other techniques.
Matching
Match the term (left) with the description (right).
| Term | Description |
|---|---|
| A: Blocking | 1: Reducing candidate pairs by partitioning on a key |
| B: One-hot | 2: Encoding categories into binary indicator columns |
| C: UUID suffix | 3: Making table names unique to avoid collisions |
Answer
A → 1 (Blocking partitions by a key), B → 2 (One-hot encoding), C → 3 (UUID suffix for unique table names).Descriptive
Explain (in 2–4 sentences) why iterative refinement of prompts improves model outputs for data analysis tasks.
Answer
Iterative refinement narrows ambiguity by incorporating feedback from prior outputs and adding missing constraints or examples. For data tasks, this process helps the model adapt to dataset-specific quirks (column names, formats, edge cases) and yields more precise, executable code or analysis steps.Creative
Design a single prompt (one or two sentences) you would send to ChatGPT to obtain a CSV summary of the top 5 states by total sales from a pandas DataFrame named df with columns state and sales. Then provide the expected short output description.
Answer
Suggested prompt: "Given a pandas DataFrame named df with columns 'state' and 'sales', provide Python code that computes the top 5 states by total sales and writes a CSV named 'top5_states.csv' with columns state and total_sales." Expected output description: Python snippet using df.groupby('state')['sales'].sum().sort_values(ascending=False).head(5).reset_index().to_csv('top5_states.csv', index=False).Activities, Vocabulary, Homework, Practice Task
Activities
- Live lab: In Google Colab, load a small CSV, ask ChatGPT to generate a summarizing function, inspect the returned code, and run it after validation.
- Pair exercise: Exchange prompt templates and test them on sample datasets; record variations and results.
Vocabulary
- Prompt engineering — designing inputs to guide AI outputs.
- Blocking — partitioning records to reduce comparisons (record linkage).
- UUID suffix — a short unique identifier appended to names for isolation.
- One-hot encoding — binary representation of categorical values.
Homework
- Write three prompts for the same task (data summary) with increasing specificity. Run them (or simulate) and note differences in outputs.
- Implement the execute_query pattern with UUID-suffixed table names in a small SQLite file; include sanity checks that prevent executing DROP statements from model-generated SQL.
Practice Task
Using a small synthetic dataset (10–50 rows) containing columns date, state, surname, and sales, do the following:
- Write a prompt to ask ChatGPT for a pandas pipeline that: normalizes state names, groups sales by state, and returns the top 3 states.
- Manually review and sanitize the returned code, run it, and save the output to CSV.
- Document the prompt, the model output, the sanitization steps, and the final CSV.
Answer — Marking guide & hints
Hints: Provide sample rows in the prompt, ask explicitly for no destructive operations, and request that the model include comments. Sanitize by checking for suspicious imports or os/system calls, ensure table and file paths are safe, and run in an isolated environment (e.g., Colab). Expected deliverable: prompt text, generated code, short list of edits, and resulting CSV (3 rows).Answer
1) Generate sanitized_table_name = f"{base}_{uuid.uuid4().hex[:8]}". 2) Use parameterized queries where possible; avoid raw string interpolation for user-supplied values. 3) Validate final SQL for prohibited statements (DROP, DELETE without WHERE, ALTER). 4) Log original prompt, sanitized prompt, generated SQL, and execution result.End of lesson. For further reading: practice with pandas, recordlinkage, sqlite3, and controlled use of OpenAI / ChatGPT APIs. Always validate AI-generated code before executing it.