I haven't found it that useful for doing any actual "agentic" coding at $DAYJOB with lots of legacy code it wasn't trained on (because proprietary). I do find it useful for summarizing sections of code that I am working on and asking for snippets that do very specific things. Also, it is pretty good at writing one-off short scripts with easily definable inputs and outputs.
I have come to the conclusion that people using AI for coding need to think about it as basically an automated version of the Docs -> Copy Paste -> Stack Overflow -> Copy Paste -> Compile Error -> Google -> Copy Paste -> New feature request from management -> Random internet blog -> Copy Paste loop that most of us do for a lot of the non-logic heavy portions (e.g. API interfacing) of our work with less randomness and more pattern matching or statistics or whatever guiding the process. Honestly, pretty useful, not knocking it.
I do think there is a killer application for AI, which it is already useful for, and that the industry doesn't really promote. That is basically taking a massive amount of unstructured data on a topic and allowing people an easy way to learn from that data without having to read through all of it (which may not even be possible for a single person in their lifetime). This would be a huge boon to humanity alone given the scale of data we produce. I think fundamentally, LLMs cannot take the data that they are so good at summarizing and use it in a creative way, it looks kinda like they can, because they are so good at "borrowing" other people's creative work, but in real-world scenarios where change is constant and the external forces of today are not understood by a model that was trained 3 months ago, they fall on their faces again and again.
I think AI companies know this but cannot admit that this ground breaking (I would argue) technology might only be transformative for one half of the observe->act workflow that would be necessary to replace humans as workers because.
1. It is possible the economics don't work out without replacing workers
2. If they admitted that the only value of their tech was in distilling value already present in other people's creative work, work that the LLMs cannot create on their own, a sane government might force them to pay for their inputs.
Since everyone else is sharing their experiences with this book here is mine:
Reading this books was a huge turning point for me as someone with diagnosed mild Autism. I think a lot of the things in this books are fairly obvious to non neuro-divergent folks. But for me, it was like a manual on how to handle myself in social situations, a thing that was mysterious and frustrating to me before. I wouldn't say I am now some sort of socialite, but I am far from the days of being being excluded from basically every social group I attempted to be part of.
"Our goal should be to give an LLM coding agent zero degrees of freedom"
Wouldn't that just be called inventing a new language with all the overhead of the languages we already have? Are we getting to the point where getting LLMs to be productive and also write good code is going to require so much overhead and additional procedures and tools that we might as well write the code ourselves. Hmmm...
Actually, no. We always needed good checks - that's why you have techniques like automated canary analysis, extensive testing, checking for coverage - these are forms of "executable oracles". If you wanted to be able to do continuous deployment - you had to be very thorough in your validation.
LLMs just take this to the extreme. You can no longer rely on human code reviews (well you can but you give away all the LLM advantages) so then if you take out "human judgement" *from validation*[1], you have to resort to very sophisticated automated validation. This is it - it's not about "inventing a new language", it's about being much more thorough (and innovative, and efficient) in the validation process.
[1] never from design, or specification - you shouldn't outsource that to AI, I don't think we're close to an AI that can do that even moderately effective without human help.
If the LLM generates code exactly matching a specification, the specification becomes a conventional programing language. The LLM is just transforming from one language to another.
Been saying this for a while now. I work in aerospace, and I can tell you from first hand experience software engineers don't know what designing a spec is.
Aero, mechanical, and electrical engineers spend years designing a system. Design, requirements, reviews, redesign, more reviews, more requirements. Every single corner of the system is well understood before anything gets made. It's a detailed, time consuming, arduous process.
Software engineers think they can duplicate that process with a few skills and a weekend planning session with Claude Code. Because implementation is cheaper we don't have to go as hard as the mechanical and electrical folks, but to properly spec a system is still a massive amount of up front effort.
I honestly don't see how this is related? Nothing says "one shot a full system from a perfect specification", I don't think this was ever a goal (or that it will be practical to do so)
Yeah, precision LLM coding is kind of an oxymoron. English language -> codebase is essentially lossily-compressed logic by definition. The less lossy the compression becomes, the more you probably approach re-inventing programming languages. Which then means that in order to use LLMs to code, you're accepting some degree of imprecision.
Yea this feels like saying “if you give them good enough specs they’ll produce the code you want” which reduces to…writing the code yourself. Just with more steps.
Ok, I think you are going to need to explain to me why "Overconfidence resulting from ignorance" isn't exactly the same thing as "lacking metacognitive ability to understand one's own skill level". Just worded more simply
Embedded systems engineer with experience developing on bare-metal and embedded Linux platforms. I have developed commercial embedded products from PCB design all the way to market. I have my own lab equipment and am fully ready to start working remotely if the job requires it.
Location: Denver, CO USA
Remote: Remote | Hybrid | In Office
Willing to relocate: No
Technologies: C/C++, Python, Buildroot, ATTiny, ATMega, STM32
Résumé/CV: https://drive.google.com/file/d/1G5zuFuMXbzGVIIIhlLNEi5Mg6wDOQOFM/view?usp=sharing
Email: dkuntz417@gmail.com
Github: https://github.com/GhostWrench
Ooof, no hardware schematics? Also missing datasheets? This projects seems like a very tall task to me without these fundamental missing pieces. Will Google release them soon or am I looking the the wrong place?
IIUC, the plan is to manufacture something completely new that runs Pebble OS. It seems likely that at least one chip they used in the original pebble is no longer available in large quantities.
If I remember correctly (and I could be wrong) Fitbit only ever bought the software side of Pebble (and the developers effectively), gutting the company and leaving Eric with the Hardware side of things...
This is good advice but only it has been followed from the beginning and consistently throughout the development of the original code. It is applicable to large organizations with lots of resources who hire professional developers and have a lot of people who are familiar with the code that are active in code reviews and have some minimum form of documentation / agreement on what the logic flow in the code should look like (the article does not claim otherwise). But I would implore those who work at the 80% of other companies that this advice is nearly useless and YMMV trying to follow it. The one thing that I think is universally good advice is to try and aggressively remove code whenever possible.
Maybe this is what you are looking for? https://github.com/GhostWrench/ttrpg9000. Full disclosure: this is my design and I do sell it on Etsy (Unfortunately I am currently out of stock and won't be back in until I make some BOM simplifications for production). But if you want to make your own, you can get the parts for ~$80 USD for small runs, assuming you can solder 0805 SMD parts.
I have come to the conclusion that people using AI for coding need to think about it as basically an automated version of the Docs -> Copy Paste -> Stack Overflow -> Copy Paste -> Compile Error -> Google -> Copy Paste -> New feature request from management -> Random internet blog -> Copy Paste loop that most of us do for a lot of the non-logic heavy portions (e.g. API interfacing) of our work with less randomness and more pattern matching or statistics or whatever guiding the process. Honestly, pretty useful, not knocking it.
I do think there is a killer application for AI, which it is already useful for, and that the industry doesn't really promote. That is basically taking a massive amount of unstructured data on a topic and allowing people an easy way to learn from that data without having to read through all of it (which may not even be possible for a single person in their lifetime). This would be a huge boon to humanity alone given the scale of data we produce. I think fundamentally, LLMs cannot take the data that they are so good at summarizing and use it in a creative way, it looks kinda like they can, because they are so good at "borrowing" other people's creative work, but in real-world scenarios where change is constant and the external forces of today are not understood by a model that was trained 3 months ago, they fall on their faces again and again.
I think AI companies know this but cannot admit that this ground breaking (I would argue) technology might only be transformative for one half of the observe->act workflow that would be necessary to replace humans as workers because.
1. It is possible the economics don't work out without replacing workers 2. If they admitted that the only value of their tech was in distilling value already present in other people's creative work, work that the LLMs cannot create on their own, a sane government might force them to pay for their inputs.
reply