Most developers treat prompting as an afterthought – write something appropriate, observe the output and iterate if necessary. This approach works until reliability becomes critical. As LLMs advance into production systems, the difference between a signal usually works and one that works Continuous Becomes an engineering concern. In response, the research community has formalized incentives into a set of well-defined techniques, each designed to address a specific failure mode—be it in structure, logic, or style. These methods work entirely at the prompt layer, requiring no fine-tuning, model changes, or infrastructure upgrades.
This article focuses on five such techniques: role-specific motivation, negative sign, JSON prompting, Attentional Reasoning Questions (ARQ)And oral sampling. Rather than covering familiar baselines like zero-shots or basic thought-chains, the emphasis here is on what changes when these techniques are applied. Each is demonstrated through side-by-side comparisons on the same task, highlighting the impact on output quality and explaining the underlying mechanisms.
Here, we are setting up a minimal environment to interact with the OpenAI API. We securely load the API key at runtime using GetPass, initialize the client, and define a lightweight chat wrapper to send system and user prompts to the model (gpt-4o-mini). This keeps our experimentation cycle clean and reusable by focusing only on quick variations.
The helper functions (sections and dividers) are only for formatting the output, making it easier to compare the baseline versus the improved prompt side by side. If you don’t already have an API key, you can create one from the official dashboard here: https://platform.openai.com/api-keys
import json
from openai import OpenAI
import os
from getpass import getpass
os.environ('OPENAI_API_KEY') = getpass('Enter OpenAI API Key: ')
client = OpenAI()
MODEL = "gpt-4o-mini"
def chat(system: str, user: str, **kwargs) -> str:
"""Minimal wrapper around the chat completions endpoint."""
response = client.chat.completions.create(
model=MODEL,
messages=(
{"role": "system", "content": system},
{"role": "user", "content": user},
),
**kwargs,
)
return response.choices(0).message.content
def section(title: str) -> None:
print()
print("=" * 60)
print(f" {title}")
print("=" * 60)
def divider(label: str) -> None:
print(f"n── {label} {'─' * (54 - len(label))}")Language models are trained on a wide mix of domains – security, marketing, legal, engineering and more. When you don’t specify a role, the model pulls them all, leading to answers that are generally correct but somewhat generic. Role-specific notation fixes this by specifying a persona in the system prompt (for example, “You are a senior application security researcher”). It acts like a filter, prompting the model to respond using the language, preferences, and reasoning style of that domain.
In this example, both responses identify the XSS risk and recommend HttpOnly cookies – the underlying facts are the same. The difference lies in how the model presents the problem. Baseline treats localStorage as a configuration option with tradeoffs. The role-specific response treats this as an attack surface: it describes what an attacker can do if XSS is present, not just whether XSS is theoretically possible. That shift in framing – from “here are the risks” to “here’s what an attacker does with those risks” – is the conditioning effect in action. No new information was given. The signal simply changed which part of the model’s knowledge was given importance.
section("TECHNIQUE 1 -- Role-Specific Prompting")
QUESTION = "Our web app stores session tokens in localStorage. Is this a problem?"
baseline_1 = chat(
system="You are a helpful assistant.",
user=QUESTION,
)
role_specific = chat(
system=(
"You are a senior application security researcher specializing in "
"web authentication vulnerabilities. You think in terms of attack "
"surface, threat models, and OWASP guidelines."
),
user=QUESTION,
)
divider("Baseline")
print(baseline_1)
divider("Role-specific (security researcher)")
print(role_specific)Negative reinforcement focuses on telling the model what not to do. By default, LLMs follow the patterns learned during training and RLHF – they add friendly openings, analogies, hedging (“it depends”), and closing summaries. Although this makes responses feel helpful, it often adds unnecessary noise in technical contexts. Negative notation works by removing these defaults. Instead of describing only the desired outputs, you also restrict unwanted behaviors, which limits the output space of the model and leads to more accurate responses.
The difference in output is immediately visible. The basic response expands into a long, structured explanation with analogies, headings, and a redundant conclusion. The negatively coded version presents the same basic information in a much shorter form – direct, concise and without fillers. Nothing essential is lost; The hint simply removes the tendency to over-interpret the model and pad the response.
section("TECHNIQUE 2 -- Negative Prompting")
TOPIC = "Explain what a database index is and when you'd use one."
baseline_2 = chat(
system="You are a helpful assistant.",
user=TOPIC,
)
negative = chat(
system=(
"You are a senior backend engineer writing internal documentation.n"
"Rules:n"
"- Do NOT use marketing language or filler phrases like 'great question' or 'certainly'.n"
"- Do NOT include caveats like 'it depends' without immediately resolving them.n"
"- Do NOT use analogies unless they are necessary. If you use one, keep it to one sentence.n"
"- Do NOT pad the response -- if you've made the point, stop.n"
),
user=TOPIC,
)
divider("Baseline")
print(baseline_2)
divider("With negative prompting")
print(negative)JSON prompting becomes important when LLM output needs to be consumed by code rather than simply read by humans. Free-form responses are inconsistent – structure varies, key details are embedded in paragraphs, and small wording changes break the parsing logic. By defining the JSON schema in the prompt, you turn the structure into a hard constraint. This not only standardizes the output format but also forces the model to organize its logic into clearly defined areas like pro, con, sentiment, and rating.
The difference in output is obvious. The baseline response is readable, but unstructured – pros, cons, and emotions are mixed into the narrative text, making it difficult to parse. However, the JSON-prompted version returns clean, well-defined fields that can be loaded directly and used in code without any post-processing. Information that was previously implicit is now explicit and separated, making it easier to store, query, and compare outputs at scale.
section("TECHNIQUE 3 -- JSON Prompting")
REVIEW = """
Honestly mixed feelings about this laptop. The display is stunning -- easily the best I've
seen at this price range -- and the keyboard is surprisingly comfortable for long sessions.
Battery life, on the other hand, barely gets me through a 6-hour workday, which is
disappointing. Fan noise under load is also pretty aggressive. For light work it's great,
but I wouldn't recommend it for anyone who needs to run heavy software.
"""
SCHEMA = """
{
"overall_sentiment": "positive | negative | mixed",
"rating": ,
"pros": ("", ...),
"cons": ("", ...),
"recommended_for": "",
"not_recommended_for": ""
}
"""
baseline_3 = chat(
system="You are a helpful assistant.",
user=f"Summarize this product review:nn{REVIEW}",
)
json_output = chat(
system=(
"You are a product review parser. Extract structured information from reviews.n"
"You MUST return only a valid JSON object. No preamble, no explanation, no markdown fences.n"
f"The JSON must match this schema exactly:n{SCHEMA}"
),
user=f"Parse this review:nn{REVIEW}",
)
divider("Baseline (free-form)")
print(baseline_3)
divider("JSON prompting (raw output)")
print(json_output)
divider("Parsed & usable in code")
parsed = json.loads(json_output)
print(f"Sentiment : {parsed('overall_sentiment')}")
print(f"Rating : {parsed('rating')}/5")
print(f"Pros : {', '.join(parsed('pros'))}")
print(f"Cons : {', '.join(parsed('cons'))}")
print(f"Recommended for : {parsed('recommended_for')}")
print(f"Avoid if : {parsed('not_recommended_for')}") Attentive Reasoning Queries (ARQ) builds on the chain-of-thought inspiration, but eliminates its biggest weakness—unstructured reasoning. In standard COT, the model decides what to focus on, which may lead to gaps or irrelevant details. ARQ replaces this with a fixed set of domain-specific questions that the model must answer in sequence. This ensures that all important aspects are covered, transferring control from the model to the prompt designer. Rather than guiding how the model thinks, ARQ defines what it should think about.
In the output, the difference is visible in terms of discipline and coverage. The baseline COT response identifies key issues but moves into less relevant areas and misses in-depth analysis in some places. However, the ARQ version systematically addresses each essential point – clearly isolating vulnerabilities, handling edge cases, and evaluating performance implications. Each question acts as a checkpoint, making responses more structured, complete, and easier to audit.
section("TECHNIQUE 4 -- Attentive Reasoning Queries (ARQ)")
CODE_TO_REVIEW = """
def get_user(user_id):
query = f"SELECT * FROM users WHERE id = {user_id}"
result = db.execute(query)
return result(0) if result else None
"""
ARQ_QUESTIONS = """
Before giving your final review, answer each of the following questions in order:
Q1 (Security): Does this code have any injection vulnerabilities?
If yes, describe the exact attack vector.
Q2 (Error handling): What happens if db.execute() throws an exception?
Is that acceptable?
Q3 (Performance): Does this query retrieve more data than necessary?
What is the cost at scale?
Q4 (Correctness): Are there edge cases in the return logic that could
cause a silent bug downstream?
Q5 (Fix): Write a corrected version of the function that addresses
all issues found above.
"""
baseline_cot = chat(
system="You are a senior software engineer. Think step by step.",
user=f"Review this Python function:nn{CODE_TO_REVIEW}",
)
arq_result = chat(
system="You are a senior software engineer conducting a security-aware code review.",
user=f"Review this Python function:nn{CODE_TO_REVIEW}nn{ARQ_QUESTIONS}",
)
divider("Baseline (free CoT)")
print(baseline_cot)
divider("ARQ (structured reasoning checklist)")
print(arq_result)Verbal sampling addresses a major limitation of LLMs: they yield a single, confident answer even when multiple interpretations are possible. This happens because alignment training favors decisional outputs. As a result, the model hides its internal uncertainty. Verbal sampling fixes this by explicitly asking about multiple hypotheses with confidence rankings and supporting evidence. Rather than forcing any one answer, it presents a range of possible outcomes – all within the prompt, without requiring model changes.
In the output, it transforms the results from a single label to a structured diagnostic view. The baseline provides a classification without any indication of uncertainty. However, the verbal version lists several ranked hypotheses, each with an explanation and a way to validate or reject it. This makes the output more actionable, turning it into an aid in decision making rather than just answers. Confidence scores themselves are not exact probabilities, but they effectively indicate relative likelihood, which is often sufficient for prioritization and downstream workflows.
section("TECHNIQUE 5 -- Verbalized Sampling")
SUPPORT_TICKET = """
Hi, I set up my account last week but I can't log in anymore. I tried resetting
my password but the email never arrives. I also tried a different browser. Nothing works.
"""
baseline_5 = chat(
system="You are a support ticket classifier. Classify the issue.",
user=f"Ticket:n{SUPPORT_TICKET}",
)
verbalized = chat(
system=(
"You are a support ticket classifier.n"
"For each ticket, generate 3 distinct hypotheses about the root cause. "
"For each hypothesis:n"
" - State the category (Authentication, Email Delivery, Account State, Browser/Client, Other)n"
" - Describe the specific failure moden"
" - Assign a confidence score from 0.0 to 1.0n"
" - State what additional information would confirm or rule it outnn"
"Order hypotheses by confidence (highest first). "
"Then provide a recommended first action for the support agent."
),
user=f"Ticket:n{SUPPORT_TICKET}",
)
divider("Baseline (single answer)")
print(baseline_5)
divider("Verbalized sampling (multiple hypotheses + confidence)")
print(verbalized)check it out Full code with notebook here. Also, feel free to follow us Twitter And don’t forget to join us 130k+ ML subreddit and subscribe our newsletter. wait! Are you on Telegram? Now you can also connect with us on Telegram.
Do you need to partner with us to promote your GitHub repo or Hugging Face page or product release or webinar, etc? join us
I am a Civil Engineering graduate (2022) from Jamia Millia Islamia, New Delhi, and I have a keen interest in Data Science, especially Neural Networks and their application in various fields.
