A reader sat down with an AI tool and asked a sensible-sounding question: how do I take advantage of the step-up in basis on my IRA when I pass it to my kids?
The answer came back fluent and helpful. It explained the step-up, walked through how heirs benefit when they inherit appreciated assets, and laid out a few moves to consider. Every sentence was well formed. The tone was calm and assured.
The premise was wrong. A traditional IRA does not receive a step-up in basis at death. It is one of the few assets the tax code specifically excludes, because the money was never taxed on the way in. Heirs pay ordinary income tax on every dollar they withdraw. The reader built a question on a false foundation, and the AI did not catch it. It accepted the premise and produced a confident answer on top of it, because catching the error is not something it does.
That is the whole subject of this article. To use these tools without getting hurt, you have to understand what they are actually doing when they answer you.
What the machine is doing
Strip away the conversational surface and the mechanism is straightforward. What people call AI here is a language model. It predicts the most likely next word in a sequence, then the next word, then the next, based on patterns it absorbed from an enormous body of text. You give it a question, and it generates the continuation that is statistically most plausible given everything it has seen.
That is the entire operation. It is a prediction engine for language, running at remarkable speed and scale. Think of it as an extraordinarily well-read autocomplete. The autocomplete on your phone guesses the next word in your text message. This guesses the next word in a paragraph about your IRA, and it does so with enough sophistication that the result reads like it came from a person who understands the subject.
Why it sounds so good
The writing is fluent because the AI learned from fluent writing. It trained on books, articles, legal documents, financial explainers, and millions of other examples of people writing clearly and authoritatively. So it reliably reproduces the shape of authority: clean structure, confident tone, the right vocabulary in the right places, an answer that flows like an expert wrote it.
Those qualities are real features of the text. They are also the source of the trap. Fluency, confidence, and good structure are the signals your brain uses to judge whether someone knows what they are talking about. The AI produces all of those signals without any of the knowledge underneath them. It has learned what a correct answer sounds like, which is a different thing from producing a correct answer. The sound is reliable. The substance is not.
Why that is dangerous for money questions
In most of daily life, a confident wrong answer gets caught quickly. Ask for directions and follow them into a dead end, and reality corrects you within minutes. The feedback is fast and obvious.
Financial planning does not work that way. A wrong answer about your IRA, your trust, or your tax year can be fluent, plausible, and completely undetectable until a tax bill arrives or an account passes to the wrong place. The step-up question is a clean example. A reader who acted on that answer would build an inheritance plan around a tax benefit that does not exist for that account, and would not discover the mistake until their heirs were handed an income tax bill the plan never accounted for. The error did not announce itself. It sat quietly inside a well-written answer until the consequence came due, possibly years later.
The longer the gap between the wrong answer and the moment reality corrects it, the more expensive the wrong answer becomes. In financial decisions, that gap can be enormous.
It does not know your situation, and it cannot check
Two limits matter most, and both follow directly from how the AI works.
First, it does not have your information. It cannot see your accounts, your tax return, your trust, or the current law as it applies to your specific facts. When it answers a question about your IRA, it is not consulting your IRA. It is producing plausible language about an IRA in general, dressed in the details you happened to mention.
Second, and this is the part most people miss, it cannot tell when it is missing something. It has no awareness of the gap between what it was told and what it would need to know. A good human advisor stops and says, I need to see your return before I answer that. The AI does not, because it has no sense of what it does not have. It fills the gap with the most plausible-sounding words and presents the result with the same confidence it shows when it happens to be right.
Three things it cannot do
A real advisor, doing real work, does three things this tool cannot.
A real advisor verifies a claim against your actual documents. The AI cannot open your file, so it cannot confirm that the premise of your question is even true. It could not catch the step-up error because it never looked at anything.
A real advisor tells you the question you should have asked instead. The reader asked how to use a step-up that does not apply. The right response was to correct the question, not answer it. A person who knows the subject redirects you. The AI answers what you asked, on the terms you asked it, premise and all.
A real advisor is accountable for the outcome. When the advice is wrong, someone stands behind it, with a name, a reputation, and an obligation to you. Nothing stands behind the AI’s answer. When it is wrong, it is simply wrong, and the cost lands entirely on you.
The real risk
The danger here is not that the AI is stupid. It is genuinely impressive at what it does. The danger is that it is articulate without being accountable, and articulate without being correct, and you have no reliable way to tell the difference from inside the conversation. The wrong answer and the right answer arrive in the same confident voice, formatted the same way, equally pleasant to read. The fluency that makes the tool useful is the same fluency that hides its errors.
So use it to learn. Use it to understand a concept before a meeting, to make sense of a term, to get oriented. Those are real strengths, and a companion piece lays out exactly where the tool helps and where it has to hand off. When the moment comes to act, particularly on anything you cannot easily undo, the verifying and the deciding belong to a person who can open your file and answer for the result.
Fluent is not the same as right. The hard part is that they look identical until someone checks.
