The anxiety of working with AI to write code

Opening Dialogue: When Code Becomes a Black Box

Shiwen: Yesterday, there was a highly popular post on Hacker News where a Meta engineer said that after working with code for so many years, he felt this way for the first time. His words were roughly: When writing code now, it feels like working with a black box. He can understand the high-level logic, but he may not understand every detail. This might be becoming a new norm in the industry. What do you think?

Yongliang: I read the quote in the facts section of that post. The core concern isn’t about AI itself, but rather the change in the boundaries of “understanding.” I’ve been in tech for seventeen years and have managed teams of seventy people at most. In the past, it was assumed that code submitted by an engineer could be explained line by line. Now, this standard is shifting. It’s not so much that there’s no standard anymore, but rather that the standard has changed from “understanding every line” to “to what extent does understanding make one responsible.” This is the main topic I want to discuss in today’s episode.

Shiwen: So you’re not here to criticize AI-assisted programming?

Yongliang: No. My team has already been using AI to write tests, add comments, and modify old code. It’s indeed fast. But no matter how fast it is, who is responsible when something goes wrong? This is a question managers cannot avoid addressing. Today, I want to break down this concern from four perspectives: phenomena, personal experience, team mechanisms, and opposing boundaries.

Q1: What exactly is the engineer’s concern?

Shiwen: Let’s clarify the phenomenon first. In plain terms, what is the Meta engineer’s concern?

Yongliang: He isn’t afraid of AI writing poorly; he’s afraid of delivering a product he doesn’t fully understand. This statement, translated from the facts section, is: You understand the high level, but not every detail.

Shiwen: Is this kind of concern representative?

Yongliang: It is very representative. But it’s just an old concern in new guise. In the past, we also worried: If the library isn’t written by you, if the framework isn’t written by you, if the cloud service infrastructure isn’t written by you, why would you dare use it? The answer is that you have established a “trust hierarchy.” You trust the Linux kernel, React, AWS because they have been tested and run in real-world environments. The problem with AI-generated code isn’t that it’s always wrong, but that its “trust hierarchy” hasn’t been fully established before you merge it into the main branch.

Shiwen: So the issue isn’t that the code comes from AI, but that the trust infrastructure isn’t sufficient?

Yongliang: That’s how it can be understood. And I’d add one more point: The more experienced the engineer, the more likely they are to have this concern, because they know how big a problem unverified details can become in a production environment. Novices don’t have this burden, and sometimes that’s not bad, but the risk is also high.

Q2: How do you personally deal with this black box?

Shiwen: Now to your personal story. With seventeen years of technical experience, do you still write code line by line yourself?

Yongliang: I will first ask myself: who owns this piece of code. If it’s mine, I must understand it; if it’s AI’s, I must verify it. This is my experience, not statistics. The specific approach is divided into three layers. In the first layer, AI provides me with a function, writes a regular expression, and generates some test data. I will quickly review the input and output to ensure there is no significant semantic deviation before using it. In the second layer, when AI gives me code involving concurrency, security, or state machines, I will ask it to explain each section and run the boundary cases myself. In the third layer, if this code is to be used in the core process, I will treat it as a PR submitted by a colleague: I will reread it, add tests, and find someone to review it, rather than just accepting it because “AI says it’s okay.”

Shiwen: Are there any scenarios where you wouldn’t let AI touch?

Yongliang: Yes. There are two types. The first type is code involving finance, permissions, or privacy, such as payment verification, authentication logic, or data masking. These areas have strong rules and serious consequences; a single hallucination from AI could lead to an accident. The second type is architectural decisions that I haven’t figured out myself. If you don’t even know what you’re doing, asking AI to write code for you is like letting it think for you. What you end up with will definitely be a beautiful but flawed shell.

Shiwen: So your bottom line is “I can’t write by hand, but I can’t also not understand.”

Yongliang: Right. This principle applies to any tool. Whether you use Git, an IDE, or AI, it doesn’t change one thing: the person who ultimately signs is you, and the responsibility lies with you. AI cannot stand trial, explain, or take blame.

Q3: How do you establish a review mechanism in a team of seventy people?

Shiwen: From individual to team. You managed a team of seventy people; how does code generated by AI flow through the team?

Yongliang: First, let’s have a hard rule: code generated by AI must be marked in the submission description. This isn’t about discriminating against AI; it’s to know where the risks are during review. When I led the team, there was a specific check item for code review: “In this change, which decisions come from humans and which from tools?” If the submitter couldn’t answer, it would be sent back for re-reading.

Shiwen: But with so much code generated by AI, can every single piece be reviewed?

Yongliang: No, so we need a layered mechanism. We divided them into three levels based on the impact. The first level is changes to ordinary modules, where AI-generated code accounts for a low percentage, and they go through regular review. The second level is changes that enter core modules, which must be reviewed by at least two people, along with unit and integration tests. The third level is the parts with the greatest impact, involving finance or permissions. AI can only assist; it cannot directly generate code, and it must be written by hand and signed by someone. The boundaries of these three levels are clearly stated in the team wiki. New employees should read this first.

Shiwen: Are there any special requirements for testing?

Yongliang: Yes. For code written by AI, the test coverage must be one level higher than manual writing. This isn’t a punishment; it’s a hedge. Because you can’t understand the details, tests must guard the boundaries of behavior. Additionally, for critical path parts of AI-written code, tests cannot be generated by the same model, as explained in Edwin0’s comment: AI might write code and tests with the same mindset, perfectly hiding bugs.

Shiwen: Are some colleagues uncomfortable with this approach?

Yongliang: Of course there is. Some people say this isn’t adding burden, right? Isn’t AI meant to improve efficiency? My response is: AI improves the “written” efficiency, not the efficiency of being “reviewed, tested, and ready for launch.” If you put unverified code into systems just because it runs fast, the time spent on debugging and fixing will multiply tenfold or even hundredfold. Over the years I’ve led my team, this principle has been repeatedly verified, so I dare to include it in our processes.

Shiwen: Besides code, what about documentation, weekly reports, and design drafts?

Yongliang: The same logic applies. AI can help you explain things clearly, but it cannot take responsibility on your behalf. Every decision and choice in the documentation must have a specific person behind it. Internally, we call this the “signing system”: who proposes the idea, who signs; AI-drafted content must also be confirmed by someone before it can be released. This rule may seem formalistic, but it concretizes the concept of “responsibility.”

Q4: What are the opposing views, and where are the boundaries?

Shiwen: There are many opposing or supplementary voices on HN. Let’s pick a few representative ones.

Yongliang: I’ll relay the most popular views from the fact box. The first comes from spacedcowboy, which says: As long as it works, I’m okay with it; just like I wouldn’t check every line of code written by each colleague. But I use extensive testing and real programs as a gate. I agree with this view; it puts AI-generated code in a reasonable position: You don’t need to understand every line, but you need observable verification methods.

Shiwen: What is the opposite of this view?

Yongliang: The second comes from edwin0: It’s more concerning when AI writes code and tests simultaneously, as both can make the same mistakes. This reminder is crucial. If both code and tests are generated by the same model, testing can easily become “self-confirming prophecy”: The model writes code according to its own logic, then writes tests according to the same logic. The tests pass, but bugs are also hidden. So our approach is that AI can write code, but key tests must be written by humans or cross-validated in another way.

Shiwen: Are there other perspectives?

Yongliang: The third is Matt BrewsBytes’ analogy, saying this is actually a form of “conceptual compression,” like Rails’ ORM. Most of the time it works well, but when the database size grows, you have to return to SQL. In other words, AI-generated code is built on an abstraction layer that helps it run, but when pushed to hard limits, you still need to dive deeper. I often use this analogy myself; it reminds us: Don’t forget the underlying principles just because AI is easy to use.

Shiwen: What about the fourth view?

Yongliang: Billylo’s view leans more towards engineering practices: Using runtime observability, cross-verification, and alerts to hedge risks. This is another expression of our team’s mechanisms. You can’t eliminate all black-box risks in advance, but you can make black boxes detectable as soon as problems occur.

Shiwen: So where are the boundaries?

Yongliang: The boundary is “verifiableability.” If a module’s inputs and outputs can be clearly tested, covered by logs and monitoring, and reviewed by colleagues, then black boxes don’t matter. But if it cannot be verified, held accountable, or audited, then whether it’s AI-generated or not, it shouldn’t be directly released. This boundary has nothing to do with AI; it relates to engineering competence.

Conclusion: The black box will not disappear, but responsibility cannot be shifted

Shiwen: Let’s wrap up with a quote for today’s issue.

Yongliang: The anxiety about AI writing code is essentially a reallocation of responsibility. In the past, the responsibility was whether “you wrote it,” now it’s whether “you understand it, verify it, and can be responsible for it.” The more powerful the tools, the more valuable human judgment becomes. The black box will not disappear, but you can make it manageable through testing, review, monitoring, and signature mechanisms.

Shiwen: More specifically, what can ordinary engineers take away today?

Yongliang: Three points. First, AI can be your co-pilot, but the driving license is yours. Second, if you don’t understand the code, either figure it out or add enough testing and monitoring to prevent the black box from operating without oversight. Third, all code that enters production must ultimately be accountable to a real person. No matter how technology changes, this rule remains unchanged. Next time when AI writes a piece of code for you, don’t rush to submit it—first ask yourself: If this code causes problems tomorrow, can I explain why it’s written that way?

Information Sources

• Hacker News discussion post, October 19, 2026, Ask HN: Meta engineer Ali Hamie said, “I’m writing code these days and it all feels like a black box… You understand the high level, but not every detail. And maybe this is becoming the new standard.” (Reported facts, direct quote)
• HN high-upvote comment by spacedcowboy: “If it works, I’m fine; I don’t inspect every line of code written by every colleague, but rely on tests and real programs as gates.” (Summary of reported facts)
• HN high-upvote comment by edwin0: “AI writing both code and tests is more unsettling because they may share the same mistakes.” (Summary of reported facts)
• HN high-upvote comment by mattbrewsbytes: The concept is akin to Rails ORM; abstraction layers work most of the time, but when databases grow large, SQL is still needed. (Summary of reported facts)
• HN high-upvote comment by billylo: Risk mitigation through runtime observability, cross-verification, and alerts. (Summary of reported facts)
• Guest experience: Seventeen years of technical experience, with a peak team of seventy people; signature-based code review, AI code annotation, test coverage hedging, and dual-reviewing of core modules are all personal practices.