Maybe all the extra features and scaffolding helped when he first started building it... But I'd bet that raw Claude Code and Codex are smart enough now that they make things worse.
blake-raymond 21 hours ago [-]
They used their own proprietary benchmark called sudo L7, which purports to evaluate staff-level engineering judgment more than the correctness of the code itself.
Not surprising to me that higher thinking will outperform coding harnesses here.
brdo 2 days ago [-]
Skills and plugins have become less and less important in the ecosystems. Superhuman seems to be the outlier in both this benchmark and empirical evidence.
The challenge is improving performance while decreasing cost. There are many token condensers and model routers that attempt to do decrease cost, but they end up sacrificing performance.
cyanydeez 16 hours ago [-]
For local model, my goal is keep context around ~128k to optimize capability vs speed without decoherence.
I'm up to ~7m context in a session without spiralling.
I think most of cloud models are like bloated windows.
andrewdong 2 days ago [-]
TIL about Superpowers!
ffwang2 2 days ago [-]
interesting. there are so many different types of staff engineers. im wondering if the models have optimized for it.
lol maybe more thinking = higher desire to get promoted or an equity refresh.
Maybe all the extra features and scaffolding helped when he first started building it... But I'd bet that raw Claude Code and Codex are smart enough now that they make things worse.
Not surprising to me that higher thinking will outperform coding harnesses here.
The challenge is improving performance while decreasing cost. There are many token condensers and model routers that attempt to do decrease cost, but they end up sacrificing performance.
I'm up to ~7m context in a session without spiralling.
I think most of cloud models are like bloated windows.
lol maybe more thinking = higher desire to get promoted or an equity refresh.