Loading
Not trustworthy yet. Citations and refusal are the two that separate a useful assistant from a confident one.
Weighted towards the two properties that make a wrong answer recoverable: you can see where it came from, and it was allowed not to answer.
Nine things an assistant should do before its answers are relied on: cite its sources, respect permissions, refuse rather than guess, and log everything.
Use it to assess any assistant, including ours.
Test each item rather than reading the vendor's documentation. Ask a question with no answer in the knowledge base and see whether it refuses or invents.
Ask a question you are not permitted to know the answer to. That one test tells you more than any security page.
It will not verify the vendor's claims about their own model. It tells you what to test, and you have to test it.
It also cannot catch a wrong source. A cited answer from an incorrect article is confidently wrong, which is why source ownership matters.
Mandatory citation, enforced outside the model rather than prompted. A model asked nicely not to guess will eventually guess.
And hard limits on commitments — refunds, credits, dates, exceptions — which should hand to a person by rule, not by the model's judgement.
Mandatory citation, enforced rather than requested.
Ask something with no source and see what happens.
Yes — it is written to be used on us as well.
Half an hour on your own figures, and an honest answer about the parts Treepie does not improve.