I’ve noticed that Claude has become considerably worse over the past few weeks. Buggy code, memory failures, inconsistent work output, and regressing back to the “my bad, you’re always right” type of response when I challenge its output. When I was investigating it, I noticed that it said I was operating Opus 5.5 at a “low” setting which is why it was returning poor answers and poorly thought considerations. I never operate on low, I always operate on high. So I decided to ask Claude a series of questions at each level of effort with the same simple parameters like “where did grapes come from”, “where did potatoes come from”, so on and so forth. They generated simple and easy responses at each level, minimal comparison in detail but then again they were easy questions. I then asked more complicated and open ended questions like “how do I learn French best?” , “how do I learn German best?”, and similarly for each level. More concerningly, besides the max effort they all generated extremely similar response structures. I asked Claude to guess which level of effort I asked each question at. After I did that it returned an error. It wouldn’t tell me what the effort levels are. I asked why and it said it doesn’t know why. In short, I’m concerned that now that Anthropic is going for an IPO they are deciding to sham effort levels to low in order to save costs and boost their “financial capabilities and growth” right before the IPO. Thoughts? Read more
