Roberto Panucci and Marcelo De Callis attended InsureTech Connect in Las Vegas on behalf of PSNA.
In the previous post, we showed that Anthropic built an AI agent for claims analysis that reaches 100% accuracy when given good rules: clear manuals, up-to-date law and well-constructed test cases. But no system is right every time.
The question that matters to the legal department is a different one: when the agent makes a mistake, who is responsible and how does the insurer defend itself?
In our view, the answer comes down to four points:
- 1.The denial remains the insurer's. Using an agent does not transfer responsibility to the system or to the vendor. Law 15.040/2024 requires that a denial be express and reasoned and does not allow additional grounds to be added later. If the agent takes part in the analysis, the grounds must be recorded from the outset and be able to withstand scrutiny by the insured, the regulator and the courts.
- 2.Automated decisions and the LGPD. Article 20 of the LGPD guarantees data subjects the right to request review of decisions made solely on the basis of automated processing. In practice, the claims workflow must establish which cases go through a person, how the insured requests review and who performs it.
- 3.The decision must be provable. In a complaint, lawsuit or inspection, it is not enough to say that the agent decided. Law 15.040/2024 already goes beyond the reasoning requirement: the adjustment report is a document common to the parties (art. 82), the regulator must disclose the content of its findings when requested (art. 80) and, if coverage is denied, the insurer must give the interested party the documents that support the decision (art. 83). If an agent took part, the insurer must be able to show what it consulted, which version of the manual and of the model was used and why it reached its conclusion. The LGPD guarantees the right to know the criteria of the automated decision (art. 20) and the CDC allows the burden of proof to be reversed. There is still no specific rule on AI in insurance (Bill 2338/2023 remains stalled in the Chamber of Deputies), but at ITC, Tony Jacob, Head of Insurance at OpenAI, predicted that regulators will require this proof. Without a record, even a correct decision is left without a defense.
- 4.The vendor contract. It defines who keeps this record and for how long, who owns the manuals and the test cases, who is responsible when the agent makes a mistake and to what extent, how the insurer audits the system and what happens when the model is updated or the vendor is replaced.
For this reason, AI governance in claims is not just a technology issue. It is regulation, contract and liability. It involves legal, compliance, technical and technology teams and must be in place before the agent goes into operation, not after the first complaint.
