Prompt Octopus is a tool that allows engineers to evaluate and compare responses from over 40 different models directly in their codebase using a VSCode extension.