OCTOPACK: INSTRUCTION TUNING LARGE LANGUAGE MODELS
WE BENCHMARK COMMITPACK AGAINST OTHER NATURAL AND SYNTHETIC INSTRUCTIONS (XP3X, SELF-INSTRUCT, OASST) ON THE 16B PARAMETER STARR MODEL, AND ACHIEVE STATE-OF-THE-ART PERFORMANCE AMONG MODELS NOT TRAINED ON OPENAI OUTPUTS, ON THE HUMANEVAL PYTHON BENCHMARK (46. 2% PASS@1).