In any case, you could not do “one shot” with the oracle — you had to put a bit of effort into it. If you simply approached them and asked for a prophecy of the future (and did nothing else) you would get no meaningful response. In contemporary terminology, you needed a bit of prompting.
To return more explicitly to the current day, many people complain about the hallucinations of top LLMs, and indeed those hallucinations are still present. (o3 is much quicker than o1 pro, but probably has a higher hallucination rate.) If you ask them only once, you are more likely to get hallucinations. If you ask a follow-up, and request a correction of errors, the answer usually is better.
Almost everyone evaluates the LLMs and their hallucinations on a one-shot basis. But historically we evaluated oracles on a multi-shot basis. It would be easy for us to do that again with LLMS, and of course many users do. For the faster models the follow-up query really does not take so long.
Or just start off on the right foot. Marius recommends this prompt:
Ultra-deep thinking mode. Greater rigor, attention to detail, and multi-angle verification. Start by outlining the task and breaking down the problem into subtasks. For each subtask, explore multiple perspectives, even those that seem initially irrelevant or improbable. Purposefully attempt to disprove or challenge your own assumptions at every step. Triple-verify everything. Critically review each step, scrutinize your logic, assumptions, and conclusions, explicitly calling out uncertainties and alternative viewpoints. Independently verify your reasoning using alternative methodologies or tools, cross-checking every fact, inference, and conclusion against external data, calculation, or authoritative sources. Deliberately seek out and employ at least twice as many verification tools or methods as you typically would. Use mathematical validations, web searches, logic evaluation frameworks, and additional resources explicitly and liberally to cross-verify your claims. Even if you feel entirely confident in your solution, explicitly dedicate additional time and effort to systematically search for weaknesses, logical gaps, hidden assumptions, or oversights. Clearly document these potential pitfalls and how you’ve addressed them. Once you’re fully convinced your analysis is robust and complete, deliberately pause and force yourself to reconsider the entire reasoning chain one final time from scratch. Explicitly detail this last reflective step.
I haven’t tried it yet, but it doesn’t cost more than a simple “Control C.”
— Read on marginalrevolution.com/marginalrevolution/2025/04/xenophons-consultation-of-the-pythia.html
Leave a Reply