Prompting for lawyers - The 8 most common pitfalls

When we talk to customers and prospects, we often get asked: “If I try it myself on ChatGPT or Claude the results often disappoint, but within Emma it seems to work. Why is that?”

The answer is usually not that Emma uses some magical model that nobody else has access to. Most of the time the difference comes from context, structure and prompting. Small differences in how you ask questions can completely change the quality of the output.

In this blogpost we will take you through the most common pitfalls that cause Large Language Models (LLMs) to give bad results. Once you understand these pitfalls, getting good results from LLMs becomes much easier. In fact, you will have learned the first step towards building an Emma workspace for yourself.

1. Lack of context

The biggest mistake people make is assuming the LLM already knows what they mean. If you don’t provide enough context, the model will fill in the gaps itself. Sometimes correctly. Often incorrectly. And because the answer sounds confident, users tend to trust it more than they should.

This happens constantly in legal work. People ask: “Are there problematic contracts in this data room?” But the model has no idea what “problematic” means in the context of your deal.

A PE buyer acquiring a SaaS business worries about completely different things than a strategic buyer acquiring a manufacturing company. Context matters.

Much better results come from prompts like: “This is a buy-side M&A due diligence for a SaaS company with high customer concentration. Analyze whether the commercial contracts contain clauses that could negatively impact revenue continuity after a change of control.”

The same applies to laws and regulations. LLMs are much better at analyzing documents than recalling them from memory. Especially in legal work, uploaded source documents almost always outperform “general knowledge prompting”.

So instead of: “Does Belgian law require consent for a share sale?” Provide the relevant law, agreement or clause and ask the model to reason from there.

The more context you give, the more the model understands your frame of reference.

2. Non-specific language

If your question is vague, the model will still try to answer it. But it will often answer a different question than the one you actually meant.

As legal professionals, you are actually very well equipped to avoid this. Legal drafting exists for a reason. Precision matters.

So don’t be afraid to use legal language when prompting. The bad reputation legalese has gotten over time is somewhat undeserved in this context. Precise language helps LLMs enormously.

For example: “Look for risky clauses” is vague. But: “Identify clauses that could allow termination, consent requirements, price renegotiation, suspension of services or assignment restrictions following a direct or indirect change of control” gives the model something concrete to work with.

Specific prompts produce specific answers.

3. Assuming your first prompt will be perfect

Even if you provide good context and use precise language, your first prompt still might not be the right prompt.

Good prompting is iterative.

One of the biggest misconceptions about LLMs is that you should get the perfect answer immediately. In practice, the best results usually come from a bit of back and forth. See how the model responds. Learn where it misunderstands you. Notice which assumptions it makes. Add the missing information and try again.

Engineers have very little to teach lawyers when it comes to legal reasoning. But adopting a trial, error, learn and retry approach from engineers will dramatically improve your prompting skills.

4. Too long conversations

Many users think that keeping one very long conversation going will continuously improve the model’s understanding. In practice, the opposite often happens.

LLMs tend to get sucked into narrow rabbit holes. After a while they start anchoring on earlier assumptions and become surprisingly difficult to steer into a different direction. You will often notice that the model keeps focusing on one specific issue while ignoring newly introduced information.

Keep conversations short. Start fresh conversations when you move to a new topic or a new part of the due diligence.

5. Too many questions at once

Another very common mistake is asking the model to do too many things simultaneously.

For example: “Review all commercial contracts for change of control clauses, assignment restrictions, exclusivity provisions, unusual liability clauses, auto-renewals, pricing risks and termination rights and summarize all material buyer risks.”

This sounds efficient. But the quality usually drops significantly. The model becomes shallow. It skips issues. It spends too little attention on the things that actually matter.

Better results usually come from splitting the workflow into smaller steps. First identify the relevant clauses. Then analyze their legal implications. Then summarize the buyer risks. This is also how many professional legal AI workflows are internally structured.

Limiting your question to one small thing at a time usually gives much deeper results.

6. Asking for short answers

LLMs are trained to mimic human behavior. And humans are often wrong when forced to answer quickly. But when you ask people to reason through something step by step, accuracy usually improves significantly (link to Thinking fast and slow from Daniel Khaneman). The same applies to LLMs.

So instead of asking: “Are there change of control clauses?” Ask: “Reason step by step about whether clauses in this agreement could be triggered by a direct or indirect change of ownership.” This may sound like a small difference, but in practice it often leads to much more complete and accurate analysis.

Reasoning models already do part of this automatically, but explicitly asking the model to reason still improves results surprisingly often.

7. Assuming an LLM knows all laws and interpretations

LLMs are trained on whatever text the provider could find or buy. That means they have seen US and UK law far more often than smaller jurisdictions.

If you ask questions about niche regulations or jurisdictions with a smaller online footprint, the model may simply not have enough reliable training data. This becomes especially relevant in cross-border M&A.

Legal concepts that sound identical can mean very different things depending on the governing law. Even concepts like “good faith”, “employee”, “material adverse effect” or “change of control” can have completely different interpretations across jurisdictions.

Another issue is that users often ask leading questions without realizing it. For example:

“How broad are the change of control clauses?” This already assumes that change of control clauses exist. The model will often follow your lead and start reasoning from that assumption.

A much better prompt is: “Are there clauses that could be triggered by a direct or indirect change of ownership? If so, explain how broadly they are defined.” Neutral prompts generally produce more balanced analysis.

The more jurisdiction-specific and assumption-free your question is, the better the results usually become.

8. Assuming an LLM will correct your mistakes

LLMs are pleasers. They are trained to be helpful and agreeable. So, if you state something incorrectly, the model will often follow your lead instead of challenging you.

This becomes dangerous because LLMs are also extremely good at sounding confident. A well-written answer is not proof that the reasoning is correct.

For example: “This agreement clearly contains a broad assignment restriction. Explain the risk.” The model may happily analyze a restriction that is not even there. A much better prompt is: “Does the agreement contain assignment restrictions? If so, explain the scope and potential impact during an acquisition.”

Always verify:

1. citations
2. clause references
3. whether the clause actually exists
4. whether the cited law applies to the correct jurisdiction

Ask the model to question your assumptions, not just confirm them.

Bonus pitfall – Assuming newer models solve these problems automatically

Many people assume that switching to a newer model automatically fixes bad prompting. In practice, newer models help, but they do not solve missing context, vague instructions or poor workflows.

A smaller model with:

1. good context
2. clear instructions
3. structured prompting

can easily outperform a frontier model with vague prompts. The quality of the setup often matters more than the model itself.

Conclusion

Once you understand these pitfalls, writing good prompts becomes much easier. Over time it even becomes second nature. So treat it like a brief: iterate, refine, and push until the output earns your name on it..

At Emma we apply these same techniques to process the thousands of documents in a data room. For every single document we run tens to hundreds of checks, each built around 8 distinct validation steps. Finally, we consolidate these results over the different documents in the data room so you can see the risks a single document poses in relationship to all the information that is provided in the data room. In practice, this means we often run over 1 000 000 different prompts for an averaged sized data room.

Want to learn more about how you can solve your legal prompting challenges or how we avoid these pitfalls at Emma? Get in touch, we are happy to dive deeper.