Roberto Panucci and Marcelo De Callis attended InsureTech Connect in Las Vegas on behalf of PSNA.
In the previous post, we showed that Anthropic built an AI agent for claims analysis that reaches 100% accuracy when given good rules: clear manuals, up-to-date law and well-constructed test cases. But no system is right every time.
The question that matters to the legal team is a different one: when the agent makes a mistake, who is responsible and how does the insurer defend itself?
In our view, the answer comes down to four points:
- The denial remains the insurer's. Using an agent does not transfer responsibility to the system or to the vendor. Law 15.040/2024 requires that a denial be express and reasoned, and it does not allow additional grounds to be added later. If the agent takes part in the analysis, the reasoning must be recorded from the start and be able to stand up before the insured, the regulator and the courts.
- Automated decisions and the LGPD. Article 20 of the LGPD gives the data subject the right to request review of a decision made solely on the basis of automated processing. In practice, the claims workflow must provide for which cases go through a person, how the insured requests review and who performs it.
- The decision must be proven. In a complaint, lawsuit or inspection, it is not enough to say that the agent decided. Law 15.040/2024 already goes beyond the duty to give reasons: the adjustment report is a document common to the parties (art. 82), the regulator must disclose the content of its investigations when requested (art. 80) and, if coverage is denied, the insurer must deliver to the interested party the documents that support the decision (art. 83). If an agent took part, the insurer must be able to show what it consulted, which version of the manual and of the model was used, and why it reached its conclusion. The LGPD guarantees the right to know the criteria of the automated decision (art. 20) and the CDC allows the burden of proof to be reversed. There is still no specific rule on AI in insurance (Bill 2338/2023 remains stalled in the Chamber of Deputies), but at ITC, Tony Jacob, Head of Insurance at OpenAI, predicted that regulators will require this proof. Without a record, even a correct decision is left undefended.
- The vendor contract. It defines who keeps this record and for how long, who owns the manuals and test cases, who is liable when the agent makes a mistake and to what extent, how the insurer audits the system, and what happens when the model is updated or the vendor is replaced.
For this reason, AI governance in claims is not just a technology issue. It is a matter of regulation, contract and liability. It involves legal, compliance, technical and technology teams, and it must be in place before the agent goes into operation, not after the first complaint.
