And, in particular, GPT-3 will be very easily misled by plausible-but-incorrect inputs -- like programs written by amateur developers. I definitely wouldn't trust it to find subtle logical errors in a student's computer program, for example.