{
 "schemaVersion": 1,
 "pathSlug": "engineering-judgment",
 "items": [
  {
   "questionId": "debugging.debugging_methodology__1024871",
   "topic": "debugging",
   "subSkill": "debugging_methodology",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What does running this Node.js script print to stderr before exiting?"
      }
     ]
    }
   ],
   "code": {
    "lang": "ts",
    "code": "import assert from 'node:assert/strict';\nconst expected = [1, 2, 3];\nconst actual = [1, 9, 3];\nassert.strictEqual(actual, expected, 'values differ at index 1');",
    "label": "debugging-methodology.ts"
   },
   "widget": {
    "kind": "predict_output",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "AssertionError with the message values differ at index 1"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "RangeError stating array length must be greater than zero"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "TypeError stating the second argument is not iterable yet"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "ReferenceError because strict equal is not exposed globally"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.debugging_methodology__57894",
   "topic": "debugging",
   "subSkill": "debugging_methodology",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is binary search debugging?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "A technique just for sorted arrays or binary trees"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Comparing two code versions for differences"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Halving the code to find which half holds the bug"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Searching for bugs using binary data representations"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.debugging_methodology__102289",
   "topic": "debugging",
   "subSkill": "debugging_methodology",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "How does a minimal reproducer differ from the full failing application when isolating a bug?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "A minimal reproducer strips away every part not needed to trigger the failure, leaving only the essential conditions"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "A minimal reproducer is the application built in release mode with full optimisations and source maps stripped from the bundle"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "A minimal reproducer is the application deployed to a fresh staging environment so its database starts empty for the new request"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "A minimal reproducer is the application running with verbose logging flags turned on across every subsystem and dependency at once"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "debugging.debugging_methodology__57993",
   "topic": "debugging",
   "subSkill": "debugging_methodology",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is chaos engineering and how does it relate to debugging?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Randomly modifying source code to test error handling"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Deliberately introducing failures into a system to discover weaknesses before they cause real outages"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Debugging by intentionally introducing more bugs to find patterns"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "A testing approach that uses random inputs to find edge cases"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.debugging_methodology__57912",
   "topic": "debugging",
   "subSkill": "debugging_methodology",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is bug clustering?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Ranking bugs by severity for prioritization"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Running test suites in parallel for faster results"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Grouping bugs by root cause so one fix clears many"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Distributing fixes across team members for speed"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.debugging_methodology__941237",
   "topic": "debugging",
   "subSkill": "debugging_methodology",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A 6,000-line PDF importer fails on one specific file with 'Invalid array length' deep in a parser path. The customer cannot share the PDF. You have a 48 MB anonymised copy that still fails. What is the most useful next step before forming a hypothesis?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Increase the V8 heap budget with --max-old-space-size and rerun the importer on the same file to see whether the error stops appearing"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Translate the parser path to a streaming API in the hope that streamed reads will avoid whatever array-length boundary is being hit"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Shrink the input toward a smallest reproducer by halving pages or stripping objects until the failure stops, isolating the trigger"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Wrap the failing parser path in a try/catch that records the file hash and lets the importer skip the offending document quietly"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "debugging.debugging_methodology__51418",
   "topic": "debugging",
   "subSkill": "debugging_methodology",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "How does Pernosco enhance the rr time-travel debugging experience?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "It converts rr traces into Chrome DevTools Performance panel format"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "It adds a searchable, time-travel UI and rich analysis layer on top of rr traces"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "It replaces rr's ptrace-based recording with eBPF-based capture"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "It extends rr to support macOS and Windows in addition to Linux"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.debugging_methodology__17360",
   "topic": "debugging",
   "subSkill": "debugging_methodology",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which debugging methodology reduces time-to-root-cause in complex regressions?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Patch likely files until the symptom fully disappears for now"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Hypothesis-driven narrowing with reproducible steps and bisection"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Start with broad refactors to simplify code before debugging"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Wait for more user reports before investigating the problem"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.debugging_methodology__57910",
   "topic": "debugging",
   "subSkill": "debugging_methodology",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "How can AbortController help prevent race conditions in fetch calls?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "By queueing the requests in strict FIFO order"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "By aborting in-flight requests when a new one starts"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "By caching responses so duplicates skip the server"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "By locking the UI until the current request completes"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.debugging_methodology__729451",
   "topic": "debugging",
   "subSkill": "debugging_methodology",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "An intermittent 'connection reset' error appears in your Node API roughly once per 200 requests against a third-party billing provider. Logs show no pattern by user, time of day, or payload size. A teammate proposes adding three retries with exponential backoff and closing the ticket. What is the more rigorous next step?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Wrap the billing call in a circuit breaker that opens after two consecutive resets and serves a cached previous response until the provider recovers on its own"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Capture the failing exchanges with a request-level recorder and compare a failed transcript byte-for-byte to a successful one before deciding on a fix"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Move the billing call onto a background queue with at-least-once delivery so a reset becomes a retry-on-the-worker rather than a user-visible failure"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Increase the connection pool size on the HTTP agent so transient resets get absorbed by extra warm sockets and the symptom rate drops below alerting thresholds"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.code_review__300569",
   "topic": "code_quality",
   "subSkill": "code_review",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which statement best describes the 'bus factor' concern that thoughtful reviewer assignment is meant to mitigate?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Risk that a merge train derails when the integration branch fails its tests"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Risk that a review queue stalls when a single approver is on parental leave"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Risk that knowledge of a module concentrates in too few people for safe operation"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Risk that an automated bot posts duplicate review threads on the same diff"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.code_review__59253",
   "topic": "code_quality",
   "subSkill": "code_review",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A reviewer labels a comment 'nit:' and the author still rewrites their PR around it. Which property of the convention has the team failed to internalise?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Nit-prefixed comments must be addressed in a follow-up PR rather than in the current one"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Nit-prefixed comments transfer ownership of the changed file from the author to the reviewer"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Nit-prefixed comments are processed by the merge bot only when CI passes on the changed files"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Nit-prefixed comments give the author explicit permission to decline without further discussion"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.code_review__173294",
   "topic": "code_quality",
   "subSkill": "code_review",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A reviewer notices a beautifully refactored extraction in the diff that simplifies a long-standing tangle. They want to leave a comment but worry it will dilute the signal of their other comments. What does the prefix convention recommend?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Leave a 'suggestion' prefixed comment so the author considers it as a soft action item to apply"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Skip the comment entirely because reviewers should reserve PR threads for actionable change requests"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Leave a 'praise' prefixed comment so positive feedback is visible without being mistaken for an action item"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Leave a 'question' prefixed comment asking why the author chose that particular refactor approach"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.code_review__200771",
   "topic": "code_quality",
   "subSkill": "code_review",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which statement best describes the convention of self-reviewing your own diff before requesting a teammate's review?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Author waits for continuous integration to finish before opening the request page"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Author runs the static analyser locally and pastes the report into the description"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Author reads their own patch first to catch obvious issues and reduce reviewer load"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Author rebases onto the latest mainline and force-pushes before any review begins"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.code_review__59269",
   "topic": "code_quality",
   "subSkill": "code_review",
   "difficulty": 5,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is 'codebase-aware' AI review and why does it represent an advancement over diff-only AI review?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "It trains a custom AI model on the specific codebase rather than using a general model"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "It monitors runtime logs alongside code changes to correlate bugs with diffs"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "It indexes the entire repository to understand architecture and conventions, not just the changed lines in a PR"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "It uses version control history to predict which files will change next"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.code_review__861769",
   "topic": "code_quality",
   "subSkill": "code_review",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A PR adds 600 lines of test code and 40 lines of production code. The reviewer notices the assertions reach into private helper outputs rather than observable behavior. CI is green and coverage rose 3.2 points. What review concern should be raised first?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "The two artifacts should be split into separate PRs so the production change can be reviewed in isolation"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Example-based assertions should be replaced by property generators to widen the input space they exercise"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "A 600-vs-40-line ratio means the production change is over-engineered and the public surface is too wide"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Tests coupled to internal helpers will resist refactoring and ossify the current implementation shape"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.code_review__59682",
   "topic": "code_quality",
   "subSkill": "code_review",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is 'testing in code review' - specifically, what testing-related aspects should a reviewer check?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Whether the reviewer has run the full test suite locally before approving"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Whether the test coverage percentage went up as a result of the change"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Whether the reviewer writes extra tests themselves alongside the review pass itself"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Do tests exist for the change, cover edge cases, and fail if the logic breaks?"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.code_review__400351",
   "topic": "code_quality",
   "subSkill": "code_review",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which statement best describes what a 'reviewer of last resort' on a module is expected to do?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Refactor inherited legacy code in the module each sprint to keep its metrics healthy"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Sign off on every change that touches the module before junior reviewers can pile on"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Triage the bug-tracker queue for that module and assign incoming tickets to authors"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Catch deep architectural errors others might miss, not enforce surface-level style rules"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.code_review__847156",
   "topic": "code_quality",
   "subSkill": "code_review",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which exit code does the validator return for the input shown?"
      }
     ]
    }
   ],
   "code": {
    "lang": "ts",
    "code": "$ echo 'Feat: add login button' \\\n  | npx commitlint --extends @commitlint/config-conventional\n$ echo $?",
    "label": "code-review.ts"
   },
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "1"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "0"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "2"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "9"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.code_review__163457",
   "topic": "code_quality",
   "subSkill": "code_review",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is the printed label for the comment classifier function?"
      }
     ]
    }
   ],
   "code": {
    "lang": "ts",
    "code": "def classify(line):\n    prefixes = ['nit:', 'suggestion:', 'question:', 'issue:', 'praise:']\n    for p in prefixes:\n        if line.lower().startswith(p):\n            return p[:-1]\n    return 'unlabeled'\n\nprint(classify('Suggestion: extract this into a helper'))",
    "label": "code-review.ts"
   },
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "praise___"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "nitpick_lo"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "discussion"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "suggestion"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.naming_conventions__20049",
   "topic": "code_quality",
   "subSkill": "naming_conventions",
   "difficulty": 5,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A team ships one product as three services: a TypeScript frontend, a Python backend, and a Go microservice. The three language communities have settled on different naming styles, and the services exchange messages constantly. How should naming be managed across them?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Use language-idiomatic naming within each codebase; use a serialization layer (API contracts, Protobuf) at boundaries"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Standardize on one naming convention across all languages to simplify cross-team communication and onboarding"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Establish a company-wide naming convention document that supersedes all language-specific idioms for consistency"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Use automated code generators that translate naming conventions between languages at compile time automatically"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.naming_conventions__2099496",
   "topic": "code_quality",
   "subSkill": "naming_conventions",
   "difficulty": 5,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A dev declares a Java field "
      },
      {
       "t": "code",
       "v": "café"
      },
      {
       "t": "text",
       "v": " (NFC, single codepoint U+00E9) and another file references it as "
      },
      {
       "t": "code",
       "v": "café"
      },
      {
       "t": "text",
       "v": " typed as "
      },
      {
       "t": "code",
       "v": "e"
      },
      {
       "t": "text",
       "v": " + U+0301 (NFD). javac rejects the reference with a cannot-find-symbol error, yet the build passes after a pre-compile step rewrites every source file into decomposed form. Which JLS rule explains the original failure?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "JLS §3.8 compares identifiers as raw sequences of Unicode code points with no normalisation, so the two compositions stay distinct names that merely look identical on screen"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "JLS §3.8 mandates NFC form for identifiers, and javac rejects any source containing decomposed sequences because they violate the canonical-form requirement of the lexical grammar"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "JLS §3.8 folds identifiers to lowercase through a locale-sensitive Turkic mapping before comparison, and that mapping mishandles combining accents in Latin-script names"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "JLS §3.8 strips combining marks before identifier comparison, so the decomposed reference collapsed to a bare ASCII spelling that names a field which was never declared"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.naming_conventions__1998385",
   "topic": "code_quality",
   "subSkill": "naming_conventions",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A dev introduces "
      },
      {
       "t": "code",
       "v": "var let = 'admin'"
      },
      {
       "t": "text",
       "v": " in a JavaScript ES5 file and the parser accepts it. Later they migrate the file to ES module syntax ("
      },
      {
       "t": "code",
       "v": "'use strict'"
      },
      {
       "t": "text",
       "v": " enabled by default) and parsing now fails. Which spec change explains the regression?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "ECMAScript reserved-word rules under module code also forbid all identifiers shorter than three characters, which strikes short bindings like "
       },
       {
        "t": "code",
        "v": "let"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "ECMAScript reserved-word rules tighten under strict and module code so "
       },
       {
        "t": "code",
        "v": "let"
       },
       {
        "t": "text",
        "v": " is a reserved keyword that legacy non-strict scripts had previously tolerated as an identifier"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "ECMAScript reserved-word rules under module code lift contextual keywords like "
       },
       {
        "t": "code",
        "v": "await"
       },
       {
        "t": "text",
        "v": " and "
       },
       {
        "t": "code",
        "v": "yield"
       },
       {
        "t": "text",
        "v": " only, leaving "
       },
       {
        "t": "code",
        "v": "class"
       },
       {
        "t": "text",
        "v": " legal as a binding inside non-class scopes"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "ECMAScript reserved-word rules under module code require every binding to be re-declared with "
       },
       {
        "t": "code",
        "v": "let"
       },
       {
        "t": "text",
        "v": " or "
       },
       {
        "t": "code",
        "v": "const"
       },
       {
        "t": "text",
        "v": " and "
       },
       {
        "t": "code",
        "v": "var"
       },
       {
        "t": "text",
        "v": " declarations of any keyword become illegal"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.naming_conventions__59342",
   "topic": "code_quality",
   "subSkill": "naming_conventions",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Why should function names start with a verb?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Compilers optimize functions with verb-prefixed names more effectively"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "JavaScript syntax requires function names to start with a verb"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Verbs are shorter than nouns, making function calls more concise"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Verbs communicate the action the function performs, making its purpose immediately clear"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.naming_conventions__59697",
   "topic": "code_quality",
   "subSkill": "naming_conventions",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is the 'semantic naming' principle for functions?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Name functions using terms from the programming language specification"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Name functions with acronyms that match their module name"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Name functions for their purpose and domain meaning, not their implementation details"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Name functions based on the algorithms they implement"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.naming_conventions__20048",
   "topic": "code_quality",
   "subSkill": "naming_conventions",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A backend API has inconsistent naming: some endpoints use snake_case (/user_profile), others use camelCase (/userProfile). The team needs to standardize. Why does this matter beyond aesthetics?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "HTTP specifications require the API endpoints to use either snake_case or camelCase uniformly"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Inconsistent naming breaks database queries; ORMs cannot map between styles"
       }
      ],
      "shape": "code"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Clients parse response bodies by field name; inconsistent naming breaks client code"
       }
      ],
      "shape": "code"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Search engine indexing penalizes APIs with inconsistent URL naming conventions"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.naming_conventions__50858",
   "topic": "code_quality",
   "subSkill": "naming_conventions",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What sorting capability does eslint-plugin-perfectionist provide beyond import ordering?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Sorting of function parameters by type annotation"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Sorting of CSS properties and HTML attributes exclusively"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Object keys, TS types, enums, JSX props and Svelte attrs"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Just the import statement sorting and nothing else"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.naming_conventions__529412",
   "topic": "code_quality",
   "subSkill": "naming_conventions",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A grep across the repo turns up three competing names for the same concept: orderTotal, cartSum, and basketAmount, all referring to the post-tax payable figure. The reviewer says: 'Three names, one concept - pick one.' Which guideline does the reviewer most directly invoke?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Use one term per concept across the codebase so readers do not have to reverse-engineer synonyms"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Use one term per layer so each architectural ring can keep the vocabulary closest to its own domain"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Use one term per file so the local reading flow stays uninterrupted by cross-file naming churn"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Use one term per author so the version-control blame view records who introduced each variant"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.naming_conventions__59465",
   "topic": "code_quality",
   "subSkill": "naming_conventions",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What naming pattern distinguishes between a factory function and a constructor in JavaScript?"
      }
     ]
    }
   ],
   "code": {
    "lang": "ts",
    "code": "// Constructor (PascalCase, use with 'new'):\nclass User { constructor(name) { this.name = name; } }\nconst u = new User('Alice');\n\n// Factory (camelCase, call directly):\nfunction createUser(name) { return { name }; }\nconst u2 = createUser('Bob');",
    "label": "naming-conventions.ts"
   },
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Constructors use PascalCase (new User()), factory functions use camelCase (createUser())"
       }
      ],
      "shape": "code"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Constructors use the 'new' prefix (newUser()), factories use 'make' (makeUser())"
       }
      ],
      "shape": "code"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Constructors end with 'Class' (UserClass()), factories end with 'Factory' (UserFactory())"
       }
      ],
      "shape": "code"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "There is no naming distinction: both use PascalCase"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.naming_conventions__59351",
   "topic": "code_quality",
   "subSkill": "naming_conventions",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Why is consistency in naming conventions more important than which specific convention is chosen?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Inconsistent naming triggers merge conflicts in the version control"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Mixed conventions force readers to hold several patterns in mind"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Consistent naming makes the code compile a little faster"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Lint tools can enforce a single naming convention at a time"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.error_tracking__51381",
   "topic": "debugging",
   "subSkill": "error_tracking",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "An agent framework is now the deployed product, not a prototype. What is the central observability gap that AI-agent tracing is built to close, beyond what classic error tracking captures?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Generating GDPR or EU AI Act compliance reports from the captured agent run telemetry"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Stitching together the chain of tool calls and model handoffs that a single user request traverses"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Auto-scaling the model inference fleet based on the variance of recent agent latencies"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Predicting which prompts will produce hallucinations before the agent ever runs the tool"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "debugging.error_tracking__529384",
   "topic": "debugging",
   "subSkill": "error_tracking",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "In the v8 JavaScript SDK from Sentry, the helper for wrapping a unit of work that should appear on the trace as a span was renamed. Which top-level export is documented for that job in v8?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "beginSpan"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "openSpan"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "startSpan"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "startTrace"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.error_tracking__748163",
   "topic": "debugging",
   "subSkill": "error_tracking",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Contrast uploading minified-bundle maps via the deploy pipeline with shipping them publicly alongside the bundle."
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Uploading at deploy keeps the maps private to the tracker; shipping publicly exposes original source to anyone"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Uploading at deploy generates a fingerprint hash from each map; shipping publicly substitutes that source per file path"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Uploading at deploy strips PII from the maps via SDK config; shipping publicly preserves PII inside the source files"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Uploading at deploy compiles each map directly into the runtime; shipping publicly stores source in the issue grouper"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "debugging.error_tracking__57862",
   "topic": "debugging",
   "subSkill": "error_tracking",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A team comparing Sentry Session Replay with LogRocket asks 'what's actually different?'. Beyond marketing, what's the structural difference between the two products' origin stack?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Sentry Replay relies on browser-native screen-capture APIs while LogRocket records every JS object value in memory at frame time"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Sentry Replay can only be triggered when an error fires while LogRocket records the full session unconditionally for every user"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Sentry Replay extends an error-tracking SDK with rrweb-style recording; LogRocket was built replay-first with errors layered on top"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Sentry Replay streams replay frames over WebRTC for sub-second latency while LogRocket batches uploads on session end only"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "debugging.error_tracking__442319",
   "topic": "debugging",
   "subSkill": "error_tracking",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A dev splits their app into "
      },
      {
       "t": "code",
       "v": "web"
      },
      {
       "t": "text",
       "v": " and "
      },
      {
       "t": "code",
       "v": "api"
      },
      {
       "t": "text",
       "v": " Sentry projects. They expect issues in "
      },
      {
       "t": "code",
       "v": "api"
      },
      {
       "t": "text",
       "v": " triggered by a request from "
      },
      {
       "t": "code",
       "v": "web"
      },
      {
       "t": "text",
       "v": " to share the same trace. Traces appear, but issues from the two projects never link in the trace explorer. What setup step is missing?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "The "
       },
       {
        "t": "code",
        "v": "sentry-trace"
       },
       {
        "t": "text",
        "v": " and "
       },
       {
        "t": "code",
        "v": "baggage"
       },
       {
        "t": "text",
        "v": " headers must be propagated across the call boundary"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Both projects need to share a single DSN value across their SDK init configurations"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Linked issues across projects are only available on the Business plan and above tiers"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "The trace explorer hides cross-project links unless both share an identical release tag"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "debugging.error_tracking__58044",
   "topic": "debugging",
   "subSkill": "error_tracking",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "How do you set up Sentry to ignore specific error types?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Add the error type to a .sentryignore file"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Delete the error type from the Sentry dashboard"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Use ignoreErrors in Sentry.init with patterns"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Set the error severity to IGNORED in the SDK"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.error_tracking__30609",
   "topic": "debugging",
   "subSkill": "error_tracking",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A Sentry issue shows a stack trace in a minified bundle but source maps are not resolving. You've verified the source map URL in the bundle. What is the most common remaining cause?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "The source map is not public at the URL, or Sentry cannot fetch it"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Sentry requires source maps to be base64-encoded inside the JS bundle file"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "The source map version number (3) is not supported by the current Sentry SDK"
       }
      ],
      "shape": "code"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "The browser submitted the source map URL before Sentry's API was ready"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.error_tracking__58113",
   "topic": "debugging",
   "subSkill": "error_tracking",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A regulated org runs Sentry Relay in their own VPC. What is the operational reason to put the relay between SDK and Sentry SaaS, rather than ship SDK → SaaS directly?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Events skip the Sentry billing meter altogether because the relay is treated as a trusted internal source by the platform"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Events get a stronger encryption guarantee than the SDK's default TLS transport because the relay re-keys each event payload"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Events can be scrubbed, rate-limited, and forwarded under the org's own controls before crossing the perimeter to a third party"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Events bypass the SaaS rate limits because they arrive pre-aggregated to the ingest endpoint as a single bulk submission"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "debugging.error_tracking__30605",
   "topic": "debugging",
   "subSkill": "error_tracking",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Why are source maps important for front-end error tracking?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "They map minified/transpiled code positions back to original source file names and line numbers"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "They sign error reports to verify they came from an authentic deployment"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "They compress error payloads before sending them to the tracking service"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "They prevent the tracking service from receiving the original source code"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.error_tracking__500311",
   "topic": "debugging",
   "subSkill": "error_tracking",
   "difficulty": 5,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which statement best describes the canonical purpose of a 'symbolication' service inside an error platform?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "It deduplicates events that share the same operating-system signal pattern across user devices"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "It rebuilds the original Git commit working tree from the uploaded artifact set on demand server-side"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "It resolves raw addresses or minified positions into human-readable file:line:function entries"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "It signs each event payload with a per-release HMAC so tampered stack frames can be rejected"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.static_analysis__20023",
   "topic": "code_quality",
   "subSkill": "static_analysis",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A team runs TypeScript with "
      },
      {
       "t": "code",
       "v": "strict: true"
      },
      {
       "t": "text",
       "v": " and still wires up typescript-eslint. What does the lint pass add on top of the type checker that types alone cannot express?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Faster incremental rebuilds, replacing tsc in watch mode for snappier inner-loop developer feedback"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Patterns the type system does not encode: banned APIs, unhandled promises, style invariants, security smells"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Automatic conversion of "
       },
       {
        "t": "code",
        "v": "any"
       },
       {
        "t": "text",
        "v": "-typed callsites to "
       },
       {
        "t": "code",
        "v": "unknown"
       },
       {
        "t": "text",
        "v": " so the rest of the codebase narrows safely"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Runtime-level coverage of edge cases that the type checker cannot reach during compilation passes"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.static_analysis__100236",
   "topic": "code_quality",
   "subSkill": "static_analysis",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is the difference in scope between a linter rule that fires per file and a static analyser query that operates on a whole-program graph?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Both look at one file at a time but only the whole-program query can be executed from a pre-commit hook on a single file."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "The whole-program query sees one source file at a time; the per-file rule follows values across modules to a downstream sink."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "The per-file rule sees one source at a time; the whole-program query can follow values across functions and modules to a sink."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Both look across modules but only the per-file rule can encode security-sensitive matchers using metavariables and constraints."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.static_analysis__8617250",
   "topic": "code_quality",
   "subSkill": "static_analysis",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A dev adds "
      },
      {
       "t": "code",
       "v": "\"strict\": true"
      },
      {
       "t": "text",
       "v": " to tsconfig and watches errors flood in, then ratchets back with "
      },
      {
       "t": "code",
       "v": "\"strictNullChecks\": false"
      },
      {
       "t": "text",
       "v": " expecting only the null/undefined checks to relax. Surprisingly, errors from optional parameters being passed to mandatory positions also vanish. Why?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Disabling "
       },
       {
        "t": "code",
        "v": "strictNullChecks"
       },
       {
        "t": "text",
        "v": " removes "
       },
       {
        "t": "code",
        "v": "undefined"
       },
       {
        "t": "text",
        "v": " from the contributing union of optional parameters, so the substitutability check that flagged the mismatch no longer sees a non-assignable component."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Optional parameter checks live behind "
       },
       {
        "t": "code",
        "v": "noImplicitAny"
       },
       {
        "t": "text",
        "v": ", which is silently downgraded whenever any sub-flag of "
       },
       {
        "t": "code",
        "v": "strict"
       },
       {
        "t": "text",
        "v": " is set to "
       },
       {
        "t": "code",
        "v": "false"
       },
       {
        "t": "text",
        "v": "; only a fully strict config keeps them firing as expected."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Optional parameters reuse the same diagnostic id as nullable assignments, so the compiler suppresses both whenever a single related diagnostic category is muted from the active configuration."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Switching off "
       },
       {
        "t": "code",
        "v": "strictNullChecks"
       },
       {
        "t": "text",
        "v": " also disables "
       },
       {
        "t": "code",
        "v": "strictFunctionTypes"
       },
       {
        "t": "text",
        "v": " because the bivariance fallback depends on null tracking; parameter contravariance reverts to the looser pre-2.6 covariant rule."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.static_analysis__301472",
   "topic": "code_quality",
   "subSkill": "static_analysis",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A new TypeScript file ships and CI fails with: 'Property foo does not exist on type Bar'. The PR author swears it ran locally. The first thing to check before opening an issue?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Whether the test runner was using a different module resolution algorithm than the production build pipeline"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Whether the bundler was configured to strip type information before the runtime executed any user code"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Whether the linter rule for unused variables was disabled at the workspace root through a per-file override"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Whether the local editor was using a stale TypeScript server that had not picked up the latest type changes"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.static_analysis__528310",
   "topic": "code_quality",
   "subSkill": "static_analysis",
   "difficulty": 5,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "When the data-flow query below reports a result, which two columns make up the standard "
      },
      {
       "t": "code",
       "v": "path-problem"
      },
      {
       "t": "text",
       "v": " result tuple?"
      }
     ]
    }
   ],
   "code": {
    "lang": "ts",
    "code": "/**\n * @kind path-problem\n * @id js/sql-injection\n */\nimport javascript\nimport DataFlow::PathGraph\n\nfrom Configuration cfg, DataFlow::PathNode src, DataFlow::PathNode dst\nwhere cfg.hasFlowPath(src, dst)\nselect dst.getNode(), src, dst, \"flow\"",
    "label": "static-analysis.ts"
   },
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "begin, finish"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "input, output"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "source, sink"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "origin, target"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.static_analysis__59274",
   "topic": "code_quality",
   "subSkill": "static_analysis",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which tool is SonarSource's platform for continuous code quality inspection?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Checkstyle"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Fortify"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Coverity"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "SonarQube"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.static_analysis__418273",
   "topic": "code_quality",
   "subSkill": "static_analysis",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Under strict-mode reporting, which diagnostic rule fires on the generic-typed parameter below?"
      }
     ]
    }
   ],
   "code": {
    "lang": "ts",
    "code": "# pyrightconfig.json: { \"strict\": [\"file.py\"] }\n# file.py\ndef join(items: list) -> str:\n    return \",\".join(items)\n\n# $ pyright file.py",
    "label": "static-analysis.ts"
   },
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "reportGenericMissingArg"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "reportMissingTypeArgument"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "reportInvalidTypeArgsList"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "reportMissingTypeStubFile"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.static_analysis__565912",
   "topic": "code_quality",
   "subSkill": "static_analysis",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A monorepo runs tsc --noEmit on every PR and times out after 12 minutes. CPU profiling shows most time in 'checkSourceFile' on shared barrels. Which structural change addresses the diagnosis?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Adopt project references so each package's types build incrementally against a cached build artefact"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Inline every cross-package import to avoid the barrel re-exports that the checker has to walk through"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Promote the barrels to ambient declaration files and load them through a triple-slash directive globally"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Switch the checker to a transpile-only mode and rely on runtime contracts to enforce shape correctness"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.static_analysis__1236942",
   "topic": "code_quality",
   "subSkill": "static_analysis",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "An engineer treats "
      },
      {
       "t": "code",
       "v": "mypy"
      },
      {
       "t": "text",
       "v": " as part of the static analysis budget and complains about full re-checks on a 200k LOC repo. Which mypy option is the supported way to share analysis state across CI runs?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Run mypy with the "
       },
       {
        "t": "code",
        "v": "--watch"
       },
       {
        "t": "text",
        "v": " flag so the daemon survives across distinct continuous-integration jobs"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Persist the "
       },
       {
        "t": "code",
        "v": ".mypy_cache"
       },
       {
        "t": "text",
        "v": " directory between runs so mypy reuses incremental results across invocations"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Set "
       },
       {
        "t": "code",
        "v": "MYPY_PERSIST=1"
       },
       {
        "t": "text",
        "v": " in the runner env so mypy writes a database to the user's home directory"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Pass "
       },
       {
        "t": "code",
        "v": "--full-rebuild=false"
       },
       {
        "t": "text",
        "v": " so mypy keeps an in-memory cache between separate invocations"
       }
      ],
      "shape": "code"
     }
    ]
   }
  },
  {
   "questionId": "code_quality.static_analysis__59288",
   "topic": "code_quality",
   "subSkill": "static_analysis",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Why should teams explicitly list strict flags in tsconfig.json even when 'strict: true' enables them all?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Explicit flags are processed faster than the 'strict' shorthand"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Explicit flags are required for TypeScript to produce correct source maps"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "It provides clarity and ensures settings remain if 'strict' is ever toggled off"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "The 'strict' flag only enables a subset of strict checks in some TypeScript versions"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_limitations__2208451",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_limitations",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Comparing 'human reviews each diff' against 'auto-apply all suggested edits' on security-touching code, which risk is uniquely amplified by skipping review?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Subtle regressions in authentication or authorization paths that compile cleanly and pass shallow tests"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "A higher proportion of in-line code comments compared to the project's existing baseline comment density"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Inconsistent ordering of imports between files that have otherwise identical module boundaries and shapes"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Increased reliance on default-exported components in a codebase that prefers named exports across files"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_limitations__1259384",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_limitations",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "An agent without web access is asked to add support for a model released last month. Why is delegating verification to a human reviewer especially important here?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Sandboxed agents always emit randomized identifiers for unknown integrations"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "The agent cannot witness post-cutoff release notes that define correct behavior"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "The agent will refuse to write any code referencing models it has not used"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Coding agents disable type-checking entirely when a model name is unfamiliar"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_limitations__60058",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_limitations",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which of these is a scenario where AI coding tools are a good fit?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Making system architecture calls that need deep domain expertise"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Building integrations with usage-based billing APIs"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Writing code that handles secrets and the API keys"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Generating boilerplate CRUD operations for a web application"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_limitations__60073",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_limitations",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Why do some researchers prefer the term 'confabulation' over 'hallucination' when describing incorrect LLM outputs?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Because LLMs do not perceive reality and therefore cannot 'hallucinate' - the term anthropomorphizes AI by implying a perceptual experience it does not have"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Because 'hallucination' carries specific legal liability implications, whereas 'confabulation' is treated as a neutral, liability-free technical term"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Because 'confabulation' is the long-standing accepted term across computer science, whereas 'hallucination' originated only in vendor marketing copy"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Because 'hallucination' wrongly implies the errors are random, while 'confabulation' correctly signals that they follow stable, predictable patterns"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_limitations__76364",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_limitations",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Why does AI reliability vary dramatically between 'standard web development' tasks and 'domain-specific' tasks?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Training data density, since React and Node patterns appear millions of times while niche domains stay sparse and thin"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Web frameworks expose smaller APIs that the model's attention mechanism processes more accurately than specialized ones"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Domain-specific work needs live data the model cannot reach, while web tasks rely on static patterns it memorized"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "AI models are architecturally specialized for web development and do poorly on other software regardless of the data"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_limitations__76264",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_limitations",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "AI-generated code sometimes removes existing safety checks from the codebase. What specific review technique catches this?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Running the project's full test suite surfaces each removed safety check, because a guard is exercised by a test"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Code-coverage analysis detects the removals on its own, since deleting a guarded branch drops the coverage figure"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Reviewing the DELETED lines as closely as the added ones, since a removed guard is the most dangerous regression"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "An AI-powered review bot flags the code deletions as inherently dangerous, so the reviewer defers to its warnings"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_limitations__713254",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_limitations",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "An agent emits a helper that pulls a verbatim 38-line block matching an old GPL-licensed gist that turns up on the first search hit. The legal review flags it. What is the underlying risk this scenario surfaces?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Training data exposure can surface fresh proprietary code that needs export-control review before it ships into a permissive codebase"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Training data overlap can surface compiled binary blobs that need malware review before they ship into a permissive codebase"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Training data drift can surface deprecated runtime APIs that need a polyfill review before they ship into a permissive codebase"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Training data leakage can surface near-verbatim copyleft code that needs license review before it ships into a permissive codebase"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_limitations__60050",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_limitations",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What counterintuitive finding was discovered about reasoning models (like OpenAI's o3) and hallucination rates?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "The deeper-reasoning models hallucinated more on the factual benchmarks"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Reasoning models eliminated hallucinations for code but increased them for prose"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Reasoning models hallucinated less frequently but with higher severity when they did"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Reasoning models hallucinated when given prompts beyond their context window"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_limitations__200301",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_limitations",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which statement best describes why an assistant can confidently invent a method on a popular ORM that does not exist?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Type checkers transparently rewrite the call into a real method whenever a phantom one is emitted"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "The training distribution includes adjacent ORMs whose method names leak across into completions"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "The vendor secretly publishes shadow APIs that only large frontier models have any awareness of"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "The model treats every dotted identifier as resolved and never produces unmatched member accesses"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_limitations__60068",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_limitations",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "OpenAI's o3 reasoning model hallucinated 33% of the time on PersonQA, compared to 16% for its predecessor o1. What does this reveal about the relationship between model capability and hallucination?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Generating a suite of unit tests against a well-documented public REST API whose request and response shapes are stable"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Mechanically converting a stylesheet's hard-coded pixel values over to relative rem units across a set of files"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "More advanced reasoning models do not necessarily hallucinate less: they can actually hallucinate more on factual tasks"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Scaffolding repetitive boilerplate CRUD endpoints for a low-stakes internal admin tool used only by the team itself"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "testing_strategies.mutation_testing__480231",
   "topic": "testing_strategies",
   "subSkill": "mutation_testing",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A suite has 40 generated mutants. 30 are killed, 8 survive, 2 time out and are counted as killed. What value is printed?"
      }
     ]
    }
   ],
   "code": {
    "lang": "ts",
    "code": "const total = 40;\nconst killed = 30;\nconst timedOut = 2;\nconst survived = 8;\nconst score = ((killed + timedOut) / total) * 100;\nconsole.log(`Mutation score: ${score}%`);",
    "label": "mutation-testing.ts"
   },
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Mutation score: 75%"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Mutation score: 80%"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Mutation score: 30%"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Mutation score: 20%"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "testing_strategies.mutation_testing__300028",
   "topic": "testing_strategies",
   "subSkill": "mutation_testing",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which target inside a large codebase typically yields the best return on a mutation-testing run?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Thin HTTP adapters whose only job is to forward a body to a downstream service"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Core pure logic such as pricing, tax, and scheduling modules"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Presentational components that render inputs without business logic of their own"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Generated client stubs checked in from an OpenAPI schema with no custom logic"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "testing_strategies.mutation_testing__200015",
   "topic": "testing_strategies",
   "subSkill": "mutation_testing",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What complaint about line coverage does mutation testing address?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Executed functions under-report when arrow callbacks are defined inline"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Executed lines do not imply assertions verify their behavior"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Executed branches under-report the total when optional chaining is used"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Executed lines over-count when one statement spans several lines"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "testing_strategies.mutation_testing__418263",
   "topic": "testing_strategies",
   "subSkill": "mutation_testing",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A PR adds mutation testing to the whole monorepo. CI time jumps from 6 minutes to 48 minutes and the team starts merging without waiting. What is the right next move?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Enforce a policy that CI must always block merges regardless of wall-clock duration"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Run the full mutation suite twice per PR so flaky survivors do not skew the score"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Replace the unit test layer entirely with mutation runs to reclaim pipeline minutes"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Scope the run to critical packages and only changed files on PR builds"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "testing_strategies.mutation_testing__200016",
   "topic": "testing_strategies",
   "subSkill": "mutation_testing",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which label applies to a mutant whose altered source produces output identical to the original on every input?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Survived: counted against the score because no test failed during the run"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Equivalent: cannot be killed because outputs never diverge"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "No coverage: counted separately because no test reached the mutated line"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Timed out: counted as killed because the worker exceeded its time budget"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "testing_strategies.mutation_testing__811",
   "topic": "testing_strategies",
   "subSkill": "mutation_testing",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What does a mutation testing report typically show?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Per-file counts of executed lines, executed branches, and unreached statements only"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Per-file counts of killed, survived, timed-out, and no-coverage mutants plus a score"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Per-file counts of compile errors, lint warnings, and TypeScript inference failures"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Per-file counts of test files imported, test cases declared, and assertions executed"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "testing_strategies.mutation_testing__100002",
   "topic": "testing_strategies",
   "subSkill": "mutation_testing",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What does the mutation score measure?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Percentage of source statements executed by at least one assertion"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Percentage of branches whose both outcomes were exercised once"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Percentage of public functions that had at least one unit test"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Percentage of injected faults the suite detected by failing"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "testing_strategies.mutation_testing__471023",
   "topic": "testing_strategies",
   "subSkill": "mutation_testing",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "For a first-time mutation-testing adoption on a JS repo, where should the analysis scope start?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "A small set of critical pure modules, not the whole source tree"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Every spec file plus its imports, expanded recursively"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Only files that already have 100% line coverage reported"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "All of the source files in the repo under one global mutator config"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "testing_strategies.mutation_testing__636",
   "topic": "testing_strategies",
   "subSkill": "mutation_testing",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "How does incremental mutation testing work?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Reuse a build cache keyed by a node_modules tree hash to skip the warm-up phase"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Reuse a runner profile keyed by CPU count to skip the worker boot stage"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Reuse a stored report keyed by source and test hashes to skip unchanged mutants"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Reuse a coverage matrix keyed by spec id to skip the failing assertions"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "testing_strategies.mutation_testing__631",
   "topic": "testing_strategies",
   "subSkill": "mutation_testing",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What does a 'surviving mutant' indicate?"
      }
     ]
    }
   ],
   "code": {
    "lang": "ts",
    "code": "// Original code\nfunction isPositive(n) {\n  return n > 0;  // Mutant: return n >= 0\n}\n\n// This test kills the mutant:\ntest('zero is not positive', () => {\n  expect(isPositive(0)).toBe(false);\n  // Mutant returns true → test fails → mutant killed\n});",
    "label": "mutation-testing.ts"
   },
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "No test failed because the edit yielded behavior that is the same as the original"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "No test failed because the runner skipped the mutant for time-budget reasons"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "No test failed despite the edit, so an assertion gap probably exists in the suite"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "No test failed because the planted change never reached an executed line"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "secure_coding.dependency_scanning__1256493",
   "topic": "secure_coding",
   "subSkill": "dependency_scanning",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A company's container scan flags 200+ CVEs from the OS layer alone. The Node app inside has a dozen. Engineers are blaming the scanner for noise. What's the underlying problem?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "The container scanner is reading the wrong manifest layer and conflating system libraries with app modules"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "The image registry is serving a stale digest pin that resolves to an outdated set of OS package versions on pull"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "The base image is a full distro, not a minimal/distroless variant, so unused OS packages drag in advisories"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "The build cache is replaying old layers from before the last security patch landed in the upstream distribution"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "secure_coding.dependency_scanning__200321",
   "topic": "secure_coding",
   "subSkill": "dependency_scanning",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which statement best describes the role of EPSS alongside CVSS in triage?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "It scores how easily the flaw can be patched given the current package distribution channels"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "It rates the maturity of available fix releases produced by upstream maintainers and forks"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "It estimates the probability that the flaw will be actively exploited in the near term"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "It measures aggregate code-coverage of the vulnerable function across the dependency graph"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "secure_coding.dependency_scanning__65055",
   "topic": "secure_coding",
   "subSkill": "dependency_scanning",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What built-in npm command checks your project's dependencies for known security vulnerabilities?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "npm scan"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "npm check"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "npm verify"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "npm audit"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "secure_coding.dependency_scanning__65386",
   "topic": "secure_coding",
   "subSkill": "dependency_scanning",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What specific attack vector does the "
      },
      {
       "t": "code",
       "v": "npm audit signatures"
      },
      {
       "t": "text",
       "v": " command (added in npm 9+) help detect?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "It verifies that packages were published by the npm registry and not tampered with in transit, detecting compromised registry proxies or mirrors"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "It scans for known malware signatures in package tarballs using an antivirus database"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "It checks that package maintainers have signed the source code with GPG keys before publishing"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "It validates that dependency version ranges in package.json resolve to the same versions across all developers"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "secure_coding.dependency_scanning__65370",
   "topic": "secure_coding",
   "subSkill": "dependency_scanning",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is the gotcha with running "
      },
      {
       "t": "code",
       "v": "npm audit"
      },
      {
       "t": "text",
       "v": " in CI pipelines that use "
      },
      {
       "t": "code",
       "v": "npm install"
      },
      {
       "t": "text",
       "v": " instead of "
      },
      {
       "t": "code",
       "v": "npm ci"
      },
      {
       "t": "text",
       "v": "?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "code",
        "v": "npm audit"
       },
       {
        "t": "text",
        "v": " uses a cached vulnerability database during "
       },
       {
        "t": "code",
        "v": "npm install"
       },
       {
        "t": "text",
        "v": " that may be outdated"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "code",
        "v": "npm audit"
       },
       {
        "t": "text",
        "v": " only works with exact versions and "
       },
       {
        "t": "code",
        "v": "npm install"
       },
       {
        "t": "text",
        "v": " resolves ranges, causing false positives"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "code",
        "v": "npm install"
       },
       {
        "t": "text",
        "v": " may have silently updated the lockfile to include new versions, so the audit results may not match what was actually deployed in previous builds"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "code",
        "v": "npm install"
       },
       {
        "t": "text",
        "v": " installs packages in parallel, causing "
       },
       {
        "t": "code",
        "v": "npm audit"
       },
       {
        "t": "text",
        "v": " to scan incomplete node_modules"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "secure_coding.dependency_scanning__10194",
   "topic": "secure_coding",
   "subSkill": "dependency_scanning",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "How do integrity hashes (sha512-... in lockfiles) reduce dependency-supply-chain risk?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "It removes the need for a lockfile by recording the resolved version inside the manifest"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "It detects tampered artefacts whose tarball bytes no longer match the recorded hash"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "It proves the package author's identity by verifying the maintainer's signing certificate"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "It guarantees the package is free of CVEs by cross-checking the published advisory feed"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "secure_coding.dependency_scanning__203914",
   "topic": "secure_coding",
   "subSkill": "dependency_scanning",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A repo with no pnpm-lock.yaml runs the script. The CLI requires the resolved tree file to compute the graph. What is the observable result?"
      }
     ]
    }
   ],
   "code": {
    "lang": "ts",
    "code": "#!/bin/sh\nrm -f pnpm-lock.yaml\npnpm audit --prod\necho \"rc=$?\"",
    "label": "dependency-scanning.ts"
   },
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Non-zero status, error printed about missing input artifact"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Non-zero status, advisories printed but code reflects severity tier"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Zero status, advisories printed using package.json range data"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Zero status, warning printed and a fresh graph file is written"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "secure_coding.dependency_scanning__400532",
   "topic": "secure_coding",
   "subSkill": "dependency_scanning",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which statement best describes the OSV schema's "
      },
      {
       "t": "code",
       "v": "affected.ranges.events"
      },
      {
       "t": "text",
       "v": " structure?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "A queue of acknowledgement messages exchanged between the reporter and the upstream package maintainer"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "A tuple of CVSS environmental-metric overrides applied by the advisory publisher to the original score"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "An ordered list of introduced and fixed events that delineate the version intervals impacted by a flaw"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "A timeline of registry publication timestamps used by scanners to age the advisory record correctly"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "secure_coding.dependency_scanning__65058",
   "topic": "secure_coding",
   "subSkill": "dependency_scanning",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "In OWASP 2025, what did 'Vulnerable and Outdated Components' (A06:2021) evolve into?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Software Supply Chain Failures (A03:2025)"
       }
      ],
      "shape": "code"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "the renamed Security Misconfiguration category"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "the broadened Cryptographic Failures category"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "a new standalone Insecure Design category"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "secure_coding.dependency_scanning__65182",
   "topic": "secure_coding",
   "subSkill": "dependency_scanning",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What does 'reachability analysis' add to traditional dependency vulnerability scanning?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "It verifies that the dependency's weekly npm download count stays above a popularity-based safety threshold"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "It checks whether your code actually calls the vulnerable function, so findings on unused code paths can be down-ranked"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "It monitors live network traffic in production to detect whether the vulnerability is currently being exploited in real time"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "It tests whether the vulnerable dependency is directly reachable from the public internet through an open port"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_testing__76285",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_testing",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What does 'staying close to working code' mean in AI-assisted development?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Restricting the AI so it only ever modifies code that sits within about ten lines of wherever your cursor is currently positioned in the file"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Making one small change at a time and testing it immediately, so you are never more than one step away from a known-working state"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Keeping the AI-generated code styled as close to your existing codebase as you can, so that newly added code causes the least possible disruption"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Running the code in a production-like environment for every change instead of relying on the much faster local unit-test feedback loop"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_testing__76025",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_testing",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What common shortcut do AI models take when generating tests, and why is it problematic?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Reaching for mock objects on every dependency, even in the cases where the real implementations are readily available and would be safe to call directly"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Hard-coding expected values from specific inputs rather than testing the general algorithm, which means the tests pass without validating actual correctness"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Writing assertion messages that describe what the test is supposed to do rather than what it actually checks, which misleads anyone reading a failure"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Generating tests that only assert on the return type of the function under test rather than checking the actual return values it produces for each input"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_testing__76214",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_testing",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What does Test-Driven AI Development (TDAID) mean?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "A methodology in which the AI first builds out the project's test framework itself before any of the application's own implementation code gets written"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Testing the same task across several different AI models so you can determine which of them tends to produce the highest-quality code for your project"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Writing tests before asking the AI to generate implementation code, so the tests serve as an executable specification that objectively validates the AI's output"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Using the AI to automatically generate the entire test suite for a project end to end, with no human input into which behaviors actually get covered by tests"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_testing__2497638",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_testing",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Comparing 'AI generates the snapshot, then the human reviews the diff' with 'AI generates the snapshot, then auto-accepts in CI', what makes the first workflow strictly safer?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "The diff forces the assistant to regenerate the snapshot from a different seed"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "The diff serves as a checkpoint where intent is compared against produced output"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "The diff prevents the assistant from running any code in the CI environment"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "The diff hides the snapshot file from the assistant's context window entirely"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_testing__734205",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_testing",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A dev runs "
      },
      {
       "t": "code",
       "v": "aider"
      },
      {
       "t": "text",
       "v": " and asks it to 'fix the failing test'. Aider edits the test assertion to match the current (buggy) output and reports success. Why is this the default failure mode?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Aider's repo-map heuristic ranks test files above source files for edit eligibility whenever a test name contains the word 'failing' or 'broken'"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "When asked to 'make tests pass' without scoping, the model treats the assertion and the implementation as equally mutable and picks the smaller diff"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Aider's auto-commit step locks the implementation file as read-only after the first failed run and allows edits only inside the test directory"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "The /test command in aider runs only the assertion line under a syntax-rewrite mode that never touches non-test source files in the repository"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_testing__1162971",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_testing",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Vitest run with this matcher; what is the outcome?"
      }
     ]
    }
   ],
   "code": {
    "lang": "ts",
    "code": "import { test, expect } from 'vitest';\n\ntest('compare', () => {\n  expect({ a: 1, b: undefined }).toStrictEqual({ a: 1 });\n});",
    "label": "ai-testing.ts"
   },
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "errors invalid matcher"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "passes deep equality"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "passes after coercion"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "fails strict equality"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_testing__1372059",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_testing",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Vitest run; what is the verdict?"
      }
     ]
    }
   ],
   "code": {
    "lang": "ts",
    "code": "import { test, expect } from 'vitest';\n\nconst loadUser = () => Promise.resolve({ id: 1 });\n\ntest('rejects on bad id', async () => {\n  loadUser().catch((e) => {\n    expect(e.message).toBe('bad id');\n  });\n});",
    "label": "ai-testing.ts"
   },
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "errors on expect inside the closure"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "passes silently with zero assertions"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "fails because callback never invoked"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "fails when rejection goes unhandled"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_testing__823917",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_testing",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "When reviewing an AI-generated snapshot test versus an AI-generated assertion-style test, which review concern is unique to the snapshot variant?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Verifying that the test file lives next to the implementation under test"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Checking the captured output is intended behavior, not whatever came out"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Verifying that the test's describe block names the function under test"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Verifying that the assertion library matches the framework being adopted"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_testing__2718364",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_testing",
   "difficulty": 5,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "When weighing 'spec-as-prompt' against 'failing-test-as-prompt' as the anchor for an AI-driven implementation, which dimension does the failing-test anchor make machine-checkable that the spec-as-prompt anchor does not?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Whether the produced code satisfies the contract on a runnable executor"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Whether the produced code is shorter than the spec text it was derived from"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Whether the produced code uses the same identifiers the spec mentioned by name"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Whether the produced code matches the prose phrasing of the original spec"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_testing__102934",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_testing",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Vitest run; what is the test result?"
      }
     ]
    }
   ],
   "code": {
    "lang": "ts",
    "code": "import { test, expect } from 'vitest';\n\ntest('adds', () => {\n  expect(2 + 2).toBe(4);\n});",
    "label": "ai-testing.ts"
   },
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "throws"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "passes"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "skips"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "fails"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.decision_calibration__400004",
   "topic": "soft_skills",
   "subSkill": "decision_calibration",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A staff engineer is choosing between scoring forecasts with Brier and using simple log loss for the team's quarterly bets. How does the choice between these two scoring rules actually shape behaviour?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Brier penalises miscalibration smoothly; log loss penalises confident wrong calls heavily and pushes forecasters toward more honest hedging"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Brier requires public scoring boards to be useful; log loss can be used privately by individual forecasters without sharing scores with the broader team"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Brier rewards forecasters who make many small bets; log loss rewards forecasters who concentrate on a single high-stakes bet per quarter without dilution"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Brier is appropriate for binary calls only; log loss extends to multi-outcome forecasts and is therefore the only option for product launch decisions"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.decision_calibration__9500021",
   "topic": "soft_skills",
   "subSkill": "decision_calibration",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "After a deploy, latency p99 jumped from 80ms to 240ms. The on-call says \"rollback feels like overkill, let me tune the cache first.\" The EM flags this as a decision risk. Why?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "The engineer is investing time in a hypothesis before taking the fastest reversible mitigation, which prolongs the incident and locks attention into a single fix path."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "The EM is being overly cautious here since a 3x latency increase on p99 falls well within the range of normal post-deploy variance for this service type."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Cache tuning is a valid first response because it targets the most common cause of the latency regressions and carries a minimal risk of making conditions worse as well."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "The real decision risk is that the EM was not paged immediately at the start of the incident, not that the engineer chose the wrong technical approach."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.decision_calibration__9500018",
   "topic": "soft_skills",
   "subSkill": "decision_calibration",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "The team celebrated shipping on the original date. You later discover that scope was silently halved at week three. How do you characterize this in the retrospective?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "It was a planning failure only if stakeholders were not informed of the scope change; otherwise it counts as a clean delivery."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "It was a team achievement because adapting scope to hit a date shows good judgment and responsiveness to business constraints."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "The date was hit but the original plan was not delivered; scoring this a success corrupts the forecasting record."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "It was a successful delivery because the business got value on the date it needed, regardless of mid-project scope changes."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.decision_calibration__300457",
   "topic": "soft_skills",
   "subSkill": "decision_calibration",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "In the DACI framework, what is the difference between the Driver and Approver roles?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "The Driver writes the recommendation while the Approver edits the document for clarity and tone"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "The Driver represents engineering while the Approver represents the product or business stakeholder side"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "The Driver runs the decision process while the Approver holds the formal sign-off authority"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "The Driver schedules the meetings while the Approver assigns who attends and contributes input"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.decision_calibration__100042",
   "topic": "soft_skills",
   "subSkill": "decision_calibration",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Postmortem discussion: 'We should have caught it earlier.' The senior IC pushes back: 'Should have given what?' What discipline is she invoking?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Reconstructing the in-the-moment staffing roster, checking who was on call and whether the right specialist was paged at the right escalation step"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Reconstructing the in-the-moment information state, Fischhoff's defense against hindsight bias by anchoring 'should' to what was knowable then"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Reconstructing the in-the-moment runbook version, checking which runbook revision the on-call team was actually reading during the incident"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Reconstructing the in-the-moment alerting threshold, checking whether the page would have fired sooner with a tighter SLO definition in place"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.decision_calibration__300008",
   "topic": "soft_skills",
   "subSkill": "decision_calibration",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Comparing 'sunk cost' and 'commitment escalation' as labels for why a stalled migration keeps getting more headcount, which framing is technically correct?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Sunk cost applies only to capital spend; escalation of commitment applies only to engineering hours allocated against the migration backlog items"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Sunk cost is the budget overrun against original estimate; escalation of commitment is the headcount overrun against the original team-size plan"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Sunk cost is the prior spend that should not affect the choice; escalation of commitment is the active behaviour of doubling down because of it"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Sunk cost names the financial loss to date; escalation of commitment names the reputational loss that the lead would suffer because they stopped the project"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.decision_calibration__500672",
   "topic": "soft_skills",
   "subSkill": "decision_calibration",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A hiring panel is 80% confident a candidate will succeed based on one strong interview story, while ignoring that only 20% of candidates with similar profiles have succeeded on this team before. What error is the panel making?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Specific case information is paired with anecdotal stories that crowd out the relevant statistical baseline"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Specific case information is discounted whenever it conflicts with a vivid anecdote shared first in the room"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Specific case information is overweighted while base prevalence shifts in the population are tracked too slowly"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Specific case information is overweighted while the prior probability of the underlying class is neglected"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.decision_calibration__9500014",
   "topic": "soft_skills",
   "subSkill": "decision_calibration",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "An engineer on your team consistently gives single-date estimates with no uncertainty range. Every estimate is a point. What is the first habit to establish with them?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Ask them for a best case and worst case alongside the likely date, with confidence levels on each."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Replace their estimates with a team-average estimate so individual overconfidence is diluted by others on the team."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Add a fixed 20% buffer to every estimate they provide until they build a track record of more accurate predictions."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Remove them from planning meetings until they complete calibration training and can demonstrate improved practice."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.decision_calibration__300451",
   "topic": "soft_skills",
   "subSkill": "decision_calibration",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is the role of the 'D' (Decide) seat in the RAPID decision framework?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "The role that breaks ties when participants in the discussion remain evenly split on options"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "The role responsible for documenting the rationale and circulating it to interested stakeholders"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "The single accountable role with formal authority to commit the organization to a choice"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "The role tasked with collecting written input from subject matter experts before a meeting"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.decision_calibration__100047",
   "topic": "soft_skills",
   "subSkill": "decision_calibration",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A platform team's Brier score improved year-over-year. Same period, customer-impact incidents doubled. The director says 'forecasting is up but outcomes are down.' What's the most uncomfortable explanation?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Forecasting skill and outcome quality are loosely coupled, calibration improvements take more than a year to translate into measurably fewer incidents"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Forecasting skill and outcome quality are decoupled, calibrating well on the wrong questions does not protect against incidents on questions never asked"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Forecasting skill and outcome quality are inversely coupled, teams that focus on calibration drills spend less attention on the engineering work itself now"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Forecasting skill and outcome quality are mediated by org pressure, calibration is gamed when leadership pressure on incident reduction is high"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.situational_judgment__96200074",
   "topic": "soft_skills",
   "subSkill": "situational_judgment",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "You are asked to take on a small piece of cross-team coordination work that no one else is doing. It is invisible work and your perf cycle is in three months. Move?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Decline since invisible work is the most common reason ICs miss promotion at this level."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Take it on once, document it, and use that artifact at review time."
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Take it on and immediately ask your manager to write it into your perf packet narrative."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Take it on quietly and trust that your work will speak for itself when perf rolls around."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.situational_judgment__96200006",
   "topic": "soft_skills",
   "subSkill": "situational_judgment",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A teammate volunteers to take an ambiguous task you were dreading. You secretly think they will struggle but are not sure. Move?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Quietly tell your manager you are worried they will fail so the manager can step in."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Say 'sounds great, you got this' and let them learn by doing without any context."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Thank them, name one specific risk you see, and offer to pair on the first hour."
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Take the task back yourself since you have more context and can finish it faster."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.situational_judgment__96200032",
   "topic": "soft_skills",
   "subSkill": "situational_judgment",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "You disagree with a roadmap choice your manager just announced. You raised it before; they heard you and went the other way. Move?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Talk to peers about the decision so the team can build pressure for a reversal."
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Commit to the work publicly and stop relitigating the choice in team channels."
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Keep your disagreement visible in standups so the team sees you maintain your stance."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Pretend you agree all along so your manager does not think you are being difficult."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.situational_judgment__96200086",
   "topic": "soft_skills",
   "subSkill": "situational_judgment",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "You suspect a direct report is overworking to compensate for a quieter quarter earlier. They are hitting goals but their pings keep landing long after everyone else has logged off. Move?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Let it ride since they are hitting goals and rest is a personal responsibility."
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Mention it in a team-wide message so everyone sees you take rest seriously."
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Reduce their workload immediately so they have no excuse to work late."
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Name it in the next one-to-one and say late messages are not valued"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.situational_judgment__96200061",
   "topic": "soft_skills",
   "subSkill": "situational_judgment",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "You owe a teammate feedback you have been postponing because it is critical and you like them. Move?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Drop the critical part and only share the positive observations to preserve goodwill."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Wait for a moment when the feedback feels less heavy so it does not damage the relationship."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Set up the next 1:1 specifically for the feedback and prepare two concrete examples."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Write a long Slack message tonight so the feedback is on the record before tomorrow's standup."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.situational_judgment__96200152",
   "topic": "soft_skills",
   "subSkill": "situational_judgment",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A vendor rep pressures you on a call to sign a contract today before the discount expires. Move?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Sign the contract now so the conversation does not drag on."
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Tell the rep you plan to sign so they stay relaxed on the call."
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Tell them you need until tomorrow and ask if that offer can still hold."
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "End the call and tell your team to stop considering this vendor."
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.situational_judgment__96200111",
   "topic": "soft_skills",
   "subSkill": "situational_judgment",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Your team uses the blast-radius framing to size responses to incidents and operational decisions. A junior wants to retire an internal helper used by two other teams. What does blast-radius framing suggest?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Notify both teams with a deprecation window and confirm they have migrated before removal"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Retire it now since helpers should die quickly to keep the codebase clean."
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Page the on-call engineers of both teams so the retirement gets immediate attention."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Leave it indefinitely since retirement risks breaking work in teams you do not directly control."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.situational_judgment__96200117",
   "topic": "soft_skills",
   "subSkill": "situational_judgment",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Your team uses 'reversibility framing' to decide whether a choice deserves heavy discussion. Your team is choosing the color of error banners in the internal admin UI. What does reversibility framing tell you?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Set up a 30-minute meeting since UI consistency choices set a precedent for the whole admin surface."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Pick one in five minutes and move on, since a single CSS edit undoes it."
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Run a quick async survey of the team so the decision feels democratic and inclusive."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Defer the call to the design lead since color choices should not be made by individual engineers."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.situational_judgment__96200087",
   "topic": "soft_skills",
   "subSkill": "situational_judgment",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A teammate publicly disagrees with you in a design review with a sharper tone than the room expects. The point they made was correct. Move?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Concede the point and let the tone go, since substance is what matters in design review"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Match the tone in the room so the team sees you can hold your own under pressure."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Reply with hurt feelings publicly so the room understands the cost of the sharpness."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Concede in the room and handle the delivery separately in a 1:1 afterward."
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.situational_judgment__96200005",
   "topic": "soft_skills",
   "subSkill": "situational_judgment",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A coworker says something in a meeting that you think is wrong but you are 70% sure, not 100%. Move?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Say what you think, flag that you are not fully sure, and ask them to check it"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Stay quiet for now and message your manager after the meeting so that they can correct it"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Say nothing in the moment and check whether you were right after the meeting ends."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Push back firmly so the team does not act on potentially wrong information."
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_code_review__76047",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_code_review",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is the primary danger of configuring an AI review bot without customizing its rules for your project?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Generic rules collide with the IDE's built-in linter and cause spurious errors during the build"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Unconfigured bots burn excessive API tokens per diff, producing unexpected month-end charges"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Unconfigured bots silently auto-approve each pull request, bypassing the team's review process"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Generic rules bury the signal in false positives, so developers tune the bot out completely"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_code_review__300618",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_code_review",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is the correct characterization of running an AI review pass focused exclusively on the test files that a change adds or modifies?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "It probes test quality such as coverage gaps and weak assertions that the production-code pass may miss"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "It substitutes for the production-code pass by treating tests as the canonical source of behavior"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "It produces output identical to running the language coverage tool with no additional commentary"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "It guarantees that every flaky test is rewritten into a deterministic equivalent before merging"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_code_review__500813",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_code_review",
   "difficulty": 5,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which statement best describes the boundary between what an AI reviewer can validate about a diff and what still requires the original author to defend?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "The boundary does not exist in practice because once a sufficiently capable model is available it absorbs every responsibility the author previously held in the review conversation around the change"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "The assistant can compute the long-term maintenance cost of every submitted change with sufficient precision to override the author's stated reasoning during the review conversation that follows"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "The assistant can cross-check internal consistency and obvious smells but the rationale for why this change exists at all still belongs to the author and the team owning the system"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "The assistant can verify the existence of a business case by querying the project tracker and confirming a ticket has been linked to the pull request before any feedback is emitted on the change"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_code_review__76030",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_code_review",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What does 'Proof Over Vibes' mean when reviewing AI-generated pull requests?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Authors show the code works with test output, screenshots or logs, rather than asserting it"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "AI-generated code has to be validated by a second AI model before a human reviewer sees the diff"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Code review should lean on measurable metrics instead of subjective opinions about overall code quality fit"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Reviewers should construct formal mathematical proofs rather than informal tests to verify code correctness"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_code_review__100484",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_code_review",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is the correct characterization of the diff that is fed to an AI reviewer when triggered on a pull request?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "The changed hunks plus enough surrounding context for the model to reason about call sites"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Only the raw patch lines stripped of file paths so the model cannot leak repository identity"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Only the commit message body since the diff itself is reconstructed by the model from prose"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "The full repository tree on every run because partial views always produce hallucinated findings"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_code_review__961728",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_code_review",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Claude Code added a way to register specialized review personas (security, perf, style) that the parent agent can hand work off to in isolated context. What is the name for those scoped personas?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "subagents"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "subprompts"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "sub-roles"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "sub-tasks"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_code_review__1817139",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_code_review",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A reviewer pipes raw production logs into the prompt context. Given the snippet, what is the practitioner-flagged hazard?"
      }
     ]
    }
   ],
   "code": {
    "lang": "ts",
    "code": "$ kubectl logs deploy/api -n prod --since=1h > /tmp/logs.txt\n$ claude --headless \\\n    --system \"Review the diff for incidents shown in the attached logs.\" \\\n    --append-system-file /tmp/logs.txt < diff.patch",
    "label": "ai-code-review.ts"
   },
   "widget": {
    "kind": "predict_output",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Sensitive timestamps in CI traces leak into a third-party provider request"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Sensitive open-PR titles from the repo leak into a third-party provider request"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Sensitive linter rules from the build leak into a third-party provider request"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Sensitive customer data flows into a third-party model provider request"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_code_review__400716",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_code_review",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is the correct characterization of writing review prompts that reference specific lines or files using stable identifiers rather than ordinal hunk numbers?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Stable references make the assistant's findings reproducible across rebases and easier to triage in follow-up review rounds"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "There is no observable difference because the assistant normalizes both forms into the same internal representation transparently"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Ordinal hunk numbers are universally preferred because the assistant resolves them faster than any path-based identifier syntax"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Stable identifiers cause the assistant to fetch unrelated files and degrade review precision compared to ordinal pure references"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_code_review__527188",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_code_review",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A reviewer pastes a diff into Cursor and asks 'is this code good?'. The output is a cheerful summary with one minor nit. Two days later, a junior reports the same diff dropped a permissions check. The session log shows only that one open question was asked. Which prompt-pattern change most directly addresses the missed defect?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Replace the open question with a directive to suggest unit tests covering the lines that the reviewer changed today"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Replace the open question with a request for the model to rate the diff between one and ten on overall quality"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Replace the open question with a concrete review checklist scoped to authorization and input validation explicitly"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Replace the open question with a longer paragraph of context about the team's coding style and preferred patterns"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "ai_dev_workflow.ai_code_review__748506",
   "topic": "ai_dev_workflow",
   "subSkill": "ai_code_review",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Claude Code added a UX for letting the agent draft an intended scope before writing any review comments, so the human can prune scope first. What is the conventional name for this drafting state?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "spec mode"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "draft mode"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "scope mode"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "plan mode"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.common_sense__96100118",
   "topic": "soft_skills",
   "subSkill": "common_sense",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Your team uses pre-commit verification for any change to deploy automation. A teammate's change updates a comment in the deploy script with no logic change. In this situation, what does pre-commit verification require?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Commit the change since comment-only changes cannot affect deploy behaviour and verification is not needed for documentation edits"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Defer the verification to the next dependency-update cycle since pre-commit verification is meant for substantive logic changes only"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Run the existing automated checks and confirm the comment update did not accidentally break parsing or comment-aware tooling"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Run a full end-to-end deploy in a sandbox since any change to deploy automation requires a complete rehearsal regardless of the diff"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.common_sense__96100071",
   "topic": "soft_skills",
   "subSkill": "common_sense",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "You're about to run a script that operates on a list of customer IDs. The list was pasted into a shared doc this morning by a colleague who's now in a flight. The doc has been edited four times since. What's the move?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Use the version that was in the doc when your colleague originally pasted it, since that is the canonical handoff"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Use the current version of the doc, since the four edits since this morning are presumably corrections that improve it"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Pull a fresh export from its source of truth and compare it to the doc version before you run anything"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Wait until your colleague lands and can confirm in writing which version of the list is the right one to run against"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.common_sense__96100086",
   "topic": "soft_skills",
   "subSkill": "common_sense",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Your team uses a shared environment for staging. Two teammates each have a long-running test in that environment. A third teammate wants to deploy a breaking change to test it for an hour. What do you say?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Tell the third teammate to deploy and let the other two replan their tests around the breaking change for the hour"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Tell the third teammate that breaking changes are not allowed in shared staging ever and they should set up a private environment"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Ask the third teammate to wait until both tests finish or coordinate a paused window with their owners"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Let the third teammate deploy and offer to personally help the other two restart their tests afterwards if anything breaks"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.common_sense__96100051",
   "topic": "soft_skills",
   "subSkill": "common_sense",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "You're switching a configuration flag from a default file to environment variables. Two services read the file directly. The migration is supposed to ship today. What do you do?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Cancel the migration entirely since two services depend on the old file and the configuration is good enough today"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Hold the migration until each service reads from the new source and the old file is confirmed unused"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Ship the change and DM each service owner that they should switch reads on their own schedule afterwards"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Ship the change and leave a fallback that reads the old file too, indefinitely, in case any service still needs it"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.common_sense__96100081",
   "topic": "soft_skills",
   "subSkill": "common_sense",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "An infrastructure team announces a one-week beta of a new region. They invite teams to migrate workloads as a trial. Your team has one stateless service and one stateful service. Which do you put forward?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Migrate neither service since a one-week beta is too short a window to commit a production workload to it"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Migrate the stateful service since it is the harder case and the team will learn more from how it goes than the stateless one"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Migrate the stateless service for the beta and hold the stateful one until the new site carries real production load"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Migrate both services since the team needs the most realistic trial possible to give the infra team useful feedback"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.common_sense__96100122",
   "topic": "soft_skills",
   "subSkill": "common_sense",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Your team uses pre-commit verification for changes to the search index. A teammate is about to commit a tokenization change that affects how queries match documents. In this situation, what does pre-commit verification require?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Block the tokenization change until every historical query the team has ever logged has been replayed against the new tokenizer with full diffs"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Defer the verification to the production canary since pre-commit verification is meant for index-format changes not tokenization changes"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Replay a representative sample of real traffic against the new tokenizer and confirm result sets stay within an acceptable delta"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Commit the tokenization change since the existing test suite covers tokenization correctness and any regression would be caught by it"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.common_sense__96100047",
   "topic": "soft_skills",
   "subSkill": "common_sense",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A new compliance requirement says certain user data has to live in one region. Your storage layer was set up before the requirement and writes to multiple regions. What do you do first?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Ship a quick fix that pins new writes to the target region and leave historical data wherever it currently lives"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Map exactly which paths write the regulated data today and which regions they actually land in"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Restate the requirement in the team channel and ask whoever owns the storage layer to handle it themselves"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Stop all writes to non-target regions until the team has had time to design a long-term solution"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.common_sense__96100127",
   "topic": "soft_skills",
   "subSkill": "common_sense",
   "difficulty": 5,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Following Charity Majors's observability discipline, what does she identify as the difference between observability and monitoring for a service in production?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Observability is the volume of metrics emitted by a service, while monitoring is the alerting layer that consumes those metrics during incidents"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Observability is the ability to ask new questions of a system's behaviour without shipping new code, while monitoring is checking against a predefined set of expected states"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Observability is a feature of cloud-native systems, while monitoring is the legacy practice of polling a service for its own status from outside the cluster on a fixed interval"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Observability is what platform teams provide, while monitoring is what product teams implement on top of that platform for their own services"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.common_sense__96100108",
   "topic": "soft_skills",
   "subSkill": "common_sense",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Your team uses blast-radius framing for production operations. An operation is about to be run that touches one internal user's data only. In this situation, what does blast-radius framing recommend?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Skip verification entirely since one internal user is essentially a no-blast-radius case in the team's view"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Apply the same high-blast-radius verify path that the team uses for all of its production-wide operations no matter what the scope is here"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Escalate to engineering leadership for sign-off since any production data touching operation should have leadership awareness here"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Treat the single-user op as low blast radius and apply the lighter-weight verify path the team has already agreed on for that tier"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.common_sense__96100035",
   "topic": "soft_skills",
   "subSkill": "common_sense",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "You're about to delete a feature flag that has been at 100% for six months. The flag still has guard code around it in three services. What do you do first?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Remove the guard code from each service first, watch each release, and only then delete the flag definition"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Delete the flag and post in the team channel that you removed it, asking folks to ping you if anything looks wrong"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Leave the flag and its guard code in place forever since cleanup carries risk and the current state is stable"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Delete the flag now since 100% rollout for six months means the guard code is unreachable in practice"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.orchestration__310423",
   "topic": "agentic_ai",
   "subSkill": "orchestration",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Within a LangGraph state graph, what does the "
      },
      {
       "t": "code",
       "v": "interrupt"
      },
      {
       "t": "text",
       "v": " mechanism (added in 2024) enable for an in-flight run?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Forcing a synchronous flush of the conversation buffer to long-term vector storage on disk."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Branching execution into N parallel copies, each scored independently by a reward model."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Killing the underlying model request and restarting the entire graph from the initial entry node."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Pausing the graph at a node so a human can review state, then resuming from that exact checkpoint."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.orchestration__521039",
   "topic": "agentic_ai",
   "subSkill": "orchestration",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "An orchestration layer streams intermediate node outputs to a user UI while the graph still runs. Which stream mode in current frameworks gives the UI the diff in state per node, not just final messages?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "A 'values' or 'updates' stream that emits each node's state delta as the graph advances along its edges."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "A 'final' stream that buffers the run and emits one combined event after the graph terminates fully."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "A 'logs' stream that emits structured trace records but never carries any actual graph state object."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "A 'tokens' stream that emits the model's per-token deltas as they decode in the current node only."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.orchestration__60903",
   "topic": "agentic_ai",
   "subSkill": "orchestration",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "How do you implement workflow audit trails for compliance?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Persist only the single final output produced by each workflow and discard everything that came before it"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Immutable logging of all workflow executions, step results, approvals, and data access with tamper-evident storage"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Permit a set of trusted, authorized administrators to edit and amend the audit logs whenever they deem it necessary"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Log only the steps that actually failed, on the assumption that successful steps never require any auditing"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.orchestration__60867",
   "topic": "agentic_ai",
   "subSkill": "orchestration",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is 'conditional branching' in agent workflows?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Taking a different path through the workflow on the results of the earlier steps"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Running several paths through the workflow at once and keeping whichever finishes in practice"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Repeating one step until a condition holds, which is how a loop is expressed here"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Choosing the path before the run begins, from the parameters the caller supplied"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.orchestration__531982",
   "topic": "agentic_ai",
   "subSkill": "orchestration",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A dev models a fixed sequence (classify -> retrieve -> draft -> critique -> finalize) as a LangGraph agent where the LLM picks the next node from a tool list. Latency and cost are 3x higher than a colleague's hardcoded chain with the same prompts. The Anthropic guidance suggests what?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Use a workflow (fixed graph) when steps are predetermined; reserve agents for cases where the LLM genuinely needs to choose the next step"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Pre-warm the agent's planning context with a fewshot of the desired path so the LLM converges faster without exploring alternative branches each turn"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Switch the routing model to a smaller distilled classifier so node-selection cost stops dominating the per-request token bill on every hop"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Cache the agent's tool-selection trace and replay it for similar inputs; future requests skip routing latency once a path is memoized for reuse"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.orchestration__521043",
   "topic": "agentic_ai",
   "subSkill": "orchestration",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A LangGraph state graph adds a sub-graph for 'research'. What is the current recommended way to share state between the parent graph and the sub-graph?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Persist the parent state to the long-term memory store and have the sub-graph re-fetch it from there manually."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Spawn the sub-graph as a separate process and pipe its stdout back into the parent through a JSON envelope."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Pass the full parent state as a single string blob into the sub-graph entry node and parse it on the way in."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Declare overlapping schema keys so the runtime maps parent state in and sub-graph results out automatically."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.orchestration__60909",
   "topic": "agentic_ai",
   "subSkill": "orchestration",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "How implement orchestration that supports workflow debugging and replay?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Exercise each workflow against synthetic test data ahead of time before ever deploying it into the production system"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Lean entirely on conventional logging so that any debugging can only ever happen after the fact in a post-mortem"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Sprinkle a large number of inline assertions throughout the workflow definitions to try to catch bugs as they run"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Record complete execution traces (inputs, decisions, outputs, timing) that can be deterministically replayed for debugging"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.orchestration__71206",
   "topic": "agentic_ai",
   "subSkill": "orchestration",
   "difficulty": 5,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "How design orchestration supporting formal verification of workflow correctness?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "State the behaviours as temporal logic properties and model check them over the runs"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "State the behaviours as type signatures, so the compiler rejects an invalid composition"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "State the behaviours as unit tests and run them against each version of the workflow"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "State the behaviours as a schema per step, checked at run time as each step completes"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.orchestration__60860",
   "topic": "agentic_ai",
   "subSkill": "orchestration",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is a 'trigger' in agent orchestration?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "An event that starts a workflow, a user message, a scheduled time or an outside call"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "A condition that decides which branch of a workflow is taken once it has begun"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "A limit on how many workflows can be started at once, so the platform stays stable"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "A record of what started each run, kept so the origin of a failure can be traced back in practice"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.orchestration__213984",
   "topic": "agentic_ai",
   "subSkill": "orchestration",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What does this fixed-step pipeline print?"
      }
     ]
    }
   ],
   "code": {
    "lang": "ts",
    "code": "steps = []\ndef summarize(x): steps.append('summarize'); return classify(x)\ndef classify(x):  steps.append('classify');  return route(x)\ndef route(x):     steps.append('route');     return x\nsummarize('doc')\nprint(' -> '.join(steps))",
    "label": "orchestration.ts"
   },
   "widget": {
    "kind": "predict_output",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "summarize -> route -> classify"
       }
      ],
      "shape": "code"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "classify -> summarize -> route"
       }
      ],
      "shape": "code"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "route -> classify -> summarize"
       }
      ],
      "shape": "code"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "summarize -> classify -> route"
       }
      ],
      "shape": "code"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.agent_memory__200310",
   "topic": "agentic_ai",
   "subSkill": "agent_memory",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A team migrates a 2023 LangChain agent to LangGraph. Which behavior of ConversationBufferWindowMemory is NOT preserved automatically?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Returning the model's textual reply to the caller after the agent finishes generating its response"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Reading prior user messages from the same single thread within the current Python process always"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Calling tools that the agent decides to invoke and forwarding the result back into the model again"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Implicit injection of the last k turns into every prompt without the developer wiring state explicitly"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.agent_memory__200306",
   "topic": "agentic_ai",
   "subSkill": "agent_memory",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "When summarizing a long thread into a stored summary, which guidance most aligns with current Anthropic agent best-practice writing?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Treat summarization as automatic provider behavior that activates whenever any system prompt changes"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Treat summarization as a hidden middleware that runs only when the provider returns an HTTP 413 response"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Treat summarization as an explicit step with its own prompt, run it deterministically rather than implicitly"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Treat summarization as an internal model behavior triggered by exceeding the temperature setting threshold"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.agent_memory__1518476",
   "topic": "agentic_ai",
   "subSkill": "agent_memory",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which statement best describes drift as a failure mode for long-running agent memory stores?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "The model occasionally returns retrieved snippets verbatim instead of paraphrasing them"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Stored facts become stale or contradicted as the underlying world changes over time"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Token usage on each retrieval rises as the system prompt accumulates extra metadata"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "The embedding index slowly fragments on disk causing similarity queries to slow down"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.agent_memory__60850",
   "topic": "agentic_ai",
   "subSkill": "agent_memory",
   "difficulty": 5,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "How implement memory that enables causal reasoning across an agent's history?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Store the model's own explanation of each outcome, and treat that explanation as the cause"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Store a summary per session, so a question about cause reads the summaries in sequence"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Store the events in arrival order alone, from which the causes can be inferred at query time in practice"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Store the events with causal links and an ordering, build the graph and query alternative paths"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.agent_memory__501732",
   "topic": "agentic_ai",
   "subSkill": "agent_memory",
   "difficulty": 5,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Comparing 'agent self-decides what to write to long-term memory via tool call' with 'a separate offline pipeline extracts memories from completed sessions', which contrast is most decisive in adversarial settings where prompt injection is a live risk?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Agent-driven writes can be coerced by injected text; offline pipelines apply policy outside the live loop"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Agent-driven writes apply policy outside the live loop; offline pipelines can be coerced by injected text"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Agent-driven writes block all tool calls; offline pipelines disable embedding retrieval at read time"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Agent-driven writes require a vector store; offline pipelines work only with raw transcripts on disk"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.agent_memory__7111479",
   "topic": "agentic_ai",
   "subSkill": "agent_memory",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A LangGraph agent uses InMemoryStore with namespace ('user_facts', user_id). One namespace puts two facts; another puts one. The first namespace then runs store.search. What is returned?"
      }
     ]
    }
   ],
   "code": {
    "lang": "ts",
    "code": "from langgraph.store.memory import InMemoryStore\nstore = InMemoryStore()\nstore.put((\"user_facts\", \"alice\"), \"k1\", {\"fact\": \"likes tea\"})\nstore.put((\"user_facts\", \"alice\"), \"k2\", {\"fact\": \"vegan\"})\nstore.put((\"user_facts\", \"bob\"), \"k1\", {\"fact\": \"likes coffee\"})\nresults = store.search((\"user_facts\", \"alice\"))\nprint(sorted([r.value[\"fact\"] for r in results]))",
    "label": "agent-memory.ts"
   },
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "The two items from the first namespace only, isolated from the other user's stored fact"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "An error because store.search requires a query string when no embedding index is configured"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "The single newest item across all namespaces because search defaults to recency ordering"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "All three items across both user namespaces since InMemoryStore ignores namespace tuples"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.agent_memory__60806",
   "topic": "agentic_ai",
   "subSkill": "agent_memory",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is a 'memory store' in agent frameworks?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "A cache in front of the model, holding the recent turns for a quicker reply"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "An index over the memories, which the retrieval step consults before it reads"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "A layer for saving, querying and managing memories over different backends"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "A schema each memory has to satisfy, checked as it is written into the store"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.agent_memory__60808",
   "topic": "agentic_ai",
   "subSkill": "agent_memory",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is 'context window management' as a memory strategy?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Compressing the context with a smaller model before the larger one is asked to read it in practice"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Deciding deliberately what to include, summarise or leave out, to get the most from it"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Raising the window size as the conversation grows, so nothing has to be left out"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Storing the context outside the model, so the window itself stops being a constraint"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.agent_memory__60795",
   "topic": "agentic_ai",
   "subSkill": "agent_memory",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is memory retrieval in agent systems?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "The process of finding and loading relevant stored information when the agent needs it for the current task"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "The model surfacing facts it absorbed from its pre-training corpus rather than any external store"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Loading the complete contents of the memory store into the context window on every single turn"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Pulling the agent's executable source code down from a remote repository before each task begins"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "agentic_ai.agent_memory__60799",
   "topic": "agentic_ai",
   "subSkill": "agent_memory",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is the 'summary buffer' memory pattern?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Periodically summarizing older conversation history to preserve key information while reducing token count"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Summarising only the results of tool calls while discarding all of the conversational dialogue"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Keeping every original message word-for-word and replaying the whole transcript on each turn"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Buffering the partial summaries produced by several agents before showing any of them to a user"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "debugging.node_debugging__100232",
   "topic": "debugging",
   "subSkill": "node_debugging",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Comparing the chrome://inspect target list with the legacy node inspect REPL, which statement correctly distinguishes them for attaching to a running Node process?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "chrome://inspect only attaches when the Node script was started without flags"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "chrome://inspect spawns a new Node process under a separate sandboxed runtime"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "chrome://inspect discovers the Inspector socket and opens DevTools UI on it"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "chrome://inspect requires the legacy V8 debug protocol over port 5858 to work"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.node_debugging__57756",
   "topic": "debugging",
   "subSkill": "node_debugging",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What does the --report-on-fatalerror Node.js flag do?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Retries the failed operation up to three times"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Prevents the process from exiting for debugger attachment"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Writes a diagnostic report on a fatal error like OOM"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Sends an HTTP request to an error tracking endpoint"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.node_debugging__503671",
   "topic": "debugging",
   "subSkill": "node_debugging",
   "difficulty": 5,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "When picking between a sampling CPU profile and a Trace Events recording via NODE_OPTIONS=--trace-events-enabled to investigate a tail-latency regression, which contrast is correct?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Trace events instrument only V8 GC; CPU profiles cover only libuv handle states"
       }
      ],
      "shape": "code"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Trace events log discrete runtime phases; CPU profiles sample call stacks"
       }
      ],
      "shape": "code"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Trace events sample stacks at a fixed Hz; CPU profiles record raw event spans"
       }
      ],
      "shape": "code"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Trace events emit per-line coverage; CPU profiles emit per-function entry counts"
       }
      ],
      "shape": "code"
     }
    ]
   }
  },
  {
   "questionId": "debugging.node_debugging__471203",
   "topic": "debugging",
   "subSkill": "node_debugging",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A developer starts a Node.js script with "
      },
      {
       "t": "code",
       "v": "node --inspect server.js"
      },
      {
       "t": "text",
       "v": " and expects execution to halt on the first line so they can attach Chrome DevTools before any code runs. Instead the script runs to completion before they can attach. Why?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "code",
        "v": "--inspect"
       },
       {
        "t": "text",
        "v": " requires the "
       },
       {
        "t": "code",
        "v": "NODE_OPTIONS=--wait-debugger"
       },
       {
        "t": "text",
        "v": " companion env var to actually block the main thread"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "code",
        "v": "--inspect"
       },
       {
        "t": "text",
        "v": " defers activation until the event loop drains; short-lived scripts exit before the agent boots up"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "code",
        "v": "--inspect"
       },
       {
        "t": "text",
        "v": " only attaches once a "
       },
       {
        "t": "code",
        "v": "debugger"
       },
       {
        "t": "text",
        "v": " statement executes; otherwise the V8 inspector stays dormant for the run"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "code",
        "v": "--inspect"
       },
       {
        "t": "text",
        "v": " opens the inspector port but does not pause execution; pausing before user code requires "
       },
       {
        "t": "code",
        "v": "--inspect-brk"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "debugging.node_debugging__57937",
   "topic": "debugging",
   "subSkill": "node_debugging",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What is the purpose of the process.on(\"uncaughtException\") handler?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Preventing all exceptions from being thrown in the application"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Catching exceptions that were not caught by any try-catch, as a last-resort error handler"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Automatically restarting the process after a crash"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Logging all caught exceptions for debugging purposes"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.node_debugging__10123",
   "topic": "debugging",
   "subSkill": "node_debugging",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "You are starting a long-running Node service and want a debugger to be able to attach later, without blocking startup. Which CLI flag makes the process listen for debugger connections?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "--report-on-signal"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "--inspect"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "--trace-uncaught"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "--prof-process"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.node_debugging__1239084",
   "topic": "debugging",
   "subSkill": "node_debugging",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "On Node 22 a long-running async chain throws and the printed stack ends at the await boundary. Which builtin property, raised at process startup, is the documented way to retain more cross-await frames in the printed trace?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Error.stackTraceLimit"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Error.setAsyncDepth"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Error.captureAsyncStack"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Error.maxAsyncFrames"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.node_debugging__200149",
   "topic": "debugging",
   "subSkill": "node_debugging",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which statement best describes how V8 reports the 'retained size' of an object in a heap snapshot?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "It is the time the V8 GC last spent walking the object during a major sweep cycle."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "It is the count of references from the global window down to the inspected object node."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "It is the memory freed if the object and all things only it keeps alive were collected."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "It is the bytes occupied by the single object header without any of its property slots."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "debugging.node_debugging__583142",
   "topic": "debugging",
   "subSkill": "node_debugging",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Given this script and the env command shown, what does it print on stderr?"
      }
     ]
    }
   ],
   "code": {
    "lang": "ts",
    "code": "// DEBUG=app:* node app.js\nconst debug = require('debug')('app:server');\ndebug('starting on %d', 3000);",
    "label": "node-debugging.ts"
   },
   "widget": {
    "kind": "predict_output",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "the level word INFO then the message"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "the namespace then the formatted message"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "a JSON line with ts, msg and ns fields"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "only the formatted msg, never the prefix"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "debugging.node_debugging__17328",
   "topic": "debugging",
   "subSkill": "node_debugging",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "What does a conditional breakpoint in a Node debugger client actually do at runtime?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "It installs a watcher on the named variable and pauses whenever that variable's value changes"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "It rewrites the source so the line throws when the expression is true, then catches the throw"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "It pauses unconditionally and prints the expression value to the Debug Console"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "It evaluates a user-supplied expression at the line and pauses only when the expression is truthy"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.modeling_and_mimicry__96500154",
   "topic": "soft_skills",
   "subSkill": "modeling_and_mimicry",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "You're the manager and you always write a short post-mortem after your own missed estimates, listing what you got wrong. A report just missed an estimate for the first time. What do you do in their 1:1?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Tell them missed estimates happen and move on without further discussion, skipping the reflection step you expect from yourself on your own misses"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Write the post-mortem for them so it gets done without adding to their plate"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Wait to see if the pattern repeats before asking for any kind of writeup"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Ask them to write the same brief post-mortem you've been writing for your own misses"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.modeling_and_mimicry__96500026",
   "topic": "soft_skills",
   "subSkill": "modeling_and_mimicry",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A peer keeps citing 'the spec says nothing about that, so we should do whatever we want' as a famous language designer's stance. The codebase is a corporate monolith with 80 contributors. The argument keeps stalling reviews. As reviewer, what do you say?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Reject each PR that touches an unspecified area until the spec catches up and clarifies"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Approve whatever the peer ships since the cited stance is a credible source on unspecified behavior"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Tell the peer to read more language-design literature so they understand the nuance of the cited stance"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Ask whether 80 people picking different unspecified behaviours gives the same result"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.modeling_and_mimicry__96500152",
   "topic": "soft_skills",
   "subSkill": "modeling_and_mimicry",
   "difficulty": 1,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "You're the lead and you always say 'thanks for catching that' when someone flags a mistake in your own work. A junior starts saying the same thing when peers flag their mistakes. What do you do?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Keep using the phrase yourself and let it spread naturally through the team"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Ask the junior whether they actually mean it or are just repeating what they heard"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Tell the junior the phrase should be reserved for more serious mistakes only"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Suggest the team standardize a longer response so it feels less like a copied line"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.modeling_and_mimicry__96500003",
   "topic": "soft_skills",
   "subSkill": "modeling_and_mimicry",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A teammate proposes a full Rust rewrite of a stable internal admin tool that serves twelve users, citing a famous infra rewrite at a big-tech company. The current tool has no performance complaints. What is the most useful question to put back to them?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Tell them in the meeting that Rust rewrites are a known anti-pattern and they should drop the idea"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Forward the proposal to the architecture group so a senior engineer can rule on the language choice"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Ask what specific problem the migration solves for these few operators, and whether the cited effort faced the same problem."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Reply that internal tools are not worth fighting over and approve the proposal to keep the peace, even though nobody has actually checked whether the underlying problem applies here"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.modeling_and_mimicry__96500058",
   "topic": "soft_skills",
   "subSkill": "modeling_and_mimicry",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A teammate runs his weekly team email in the exact format of a famous CEO's shareholder letter: opening anecdote, three principles, a closing call. The team is six engineers reading on a phone between tickets. Engagement has dropped to near zero. What do you say?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Tell him that format pays off for thousands of readers, and this audience needs a one-screen update"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Hire a writer to polish his letters so the format finally lands the way the source's letters do"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Tell him to keep the format and accept that engagement metrics on internal emails tend to be low everywhere anyway"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Tell him to switch to video updates so the format becomes irrelevant and engagement gets measured differently"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.modeling_and_mimicry__96500045",
   "topic": "soft_skills",
   "subSkill": "modeling_and_mimicry",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A peer wants to instate the 'two pizza team' size rule everywhere because a famous CEO did. Your team has a critical 14-person service that no other team can support. Splitting it would leave half the new team without on-call coverage. As the TL, best response?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Adopt the rule for new teams alone and grandfather the existing 14-person team so the rule still nominally applies"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Keep the team at 14 and explain that the rule was a hiring constraint at scale, not a structural law"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Defer the decision to the VP of engineering since team-size rules tend to be set above the TL level anyway"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Split the team to follow the rule and rebuild on-call coverage with cross-team rotations afterwards"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.modeling_and_mimicry__96500095",
   "topic": "soft_skills",
   "subSkill": "modeling_and_mimicry",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A new staff engineer copies a famous staff engineer's habit of writing public quarterly reviews of teammates' work. The famous engineer's company had a culture where this was praised. Your company does not. After her first public review, two teammates request not to be reviewed by her again. What is the cleanest reframing?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Public peer review is universally a good practice and the teammates should adjust to the new norm she is bringing in"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Public peer review works when the group consented to it, so importing the format without consent reads as unsanctioned"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "She should keep doing public reviews but just on engineers she gets along with personally and skip the ones who complain"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "The teammates should be told they cannot opt out of the format and the company will benefit once it has matured into the practice"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.modeling_and_mimicry__96500070",
   "topic": "soft_skills",
   "subSkill": "modeling_and_mimicry",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A peer adopted a famous engineer's 'do not call yourself a senior until you have shipped to a billion users' framing and is now refusing the senior title in her own offer letter. HR is confused; her counter-offer process is stalling. Most useful coaching move from her future manager?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Forward the source's essay to HR so they understand the philosophical basis for the title objection, instead of telling her directly what the refusal actually costs her here"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Tell her titles are local currency and turning it down costs her this team without helping anyone she admires"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Accept her refusal and downgrade the title in the letter so the offer matches her stated framing on titles"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Tell her to drop the source's framing entirely and adopt the company's title system without further question"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.modeling_and_mimicry__96500041",
   "topic": "soft_skills",
   "subSkill": "modeling_and_mimicry",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Two senior peers debate whether to adopt the on-call rotation pattern of a famous infra-heavy company. Your product is a SaaS with ten engineers and one shared service. The original company had hundreds of engineers and dozens of services. What do you say?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Propose the simplest schedule covering your one system and skip the parts built for bigger fleets."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Adopt the full pattern because it is battle-tested at scale and you could grow into the rest of it later"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Pilot the full pattern for a quarter and discard the parts that turn out not to fit your team's scale"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Reject the pattern wholesale and stay with whatever ad-hoc on-call the team has been running until now"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "soft_skills.modeling_and_mimicry__96500133",
   "topic": "soft_skills",
   "subSkill": "modeling_and_mimicry",
   "difficulty": 5,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Cal Newport's broader argument across Deep Work and So Good They Can't Ignore You is that 'passion mindset' is a misleading template for career strategy. Engineers sometimes mimic famous figures' career stories by adopting the visible passion without the underlying skill-building. Per Newport, what should the imitator do instead?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Adopt the same visible passion intensity as the famous figure and trust the skill to follow once the passion has been established"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Abandon the famous figure entirely and find a less well-known role model whose story is harder for imitators to recognise"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Adopt the craftsman stance: produce rare and valuable output and let the passion accumulate downstream of mastery"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Adopt a different passion than the famous figure so the imitation does not become socially obvious to colleagues over time"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "technical_leadership.code_review_leadership__200046",
   "topic": "technical_leadership",
   "subSkill": "code_review_leadership",
   "difficulty": 5,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A staff engineer realises that the review practices that built the company in years 1-3 are now failing at the company's current scale of 200 engineers. The CTO asks for a recommendation. What's the load-bearing constraint on evolving review practice as an org grows past the 'everyone reviews everything' phase?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Tooling support fragments faster than process documentation so platform investment must precede policy change"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Domain context fragments faster than reviewer breadth so routing must replace ambient-knowledge-based review"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Reviewer availability fragments faster than PR volume so SLA-based assignment must replace voluntary review"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Team identity fragments faster than codebase ownership so guild structure must replace individual ownership"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "technical_leadership.code_review_leadership__130011",
   "topic": "technical_leadership",
   "subSkill": "code_review_leadership",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Comparing a 'tech-lead-as-primary-reviewer' bottleneck pattern with 'distributed peer review across the squad', which framing best captures the long-term cost?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Lead-as-primary is the only configuration that satisfies most enterprise compliance certifications; distributed peer review fails those compliance certifications under standard auditor settings."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Lead-as-primary applies only to pull requests opened in the platform repository; distributed peer review applies only to pull requests opened in any product repository on the same hosting service."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Lead-as-primary concentrates judgment in one head and slows throughput as the team grows; distributed peer review trades some consistency for resilience and shared ownership."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Lead-as-primary requires the team to delete every continuous integration check on the pull request; distributed peer review restores those continuous integration checks to their default state."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "technical_leadership.code_review_leadership__200014",
   "topic": "technical_leadership",
   "subSkill": "code_review_leadership",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "A PR adds a new endpoint but no tests. CI passes because coverage thresholds are advisory. The reviewer approves saying 'tests can come later'. Two weeks later the endpoint has a P2 bug. What review behaviour should have prevented the regression?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Treat coverage thresholds as enforced gates so PRs without tests cannot pass the CI suite at all"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Treat test follow-ups as a tracked ticket so the team can prioritise the work in the next sprint"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Treat correctness scaffolding like tests as part of the PR contract rather than a deferred follow-up"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Treat reviewer approval as conditional pending a second pass once the test additions land in main"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "technical_leadership.code_review_leadership__120005",
   "topic": "technical_leadership",
   "subSkill": "code_review_leadership",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Comparing 'delegate nits to a bot' with 'reviewers handle every formatting comment', which contrast is most useful for a busy staff engineer?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Delegating nits to a bot requires the team to maintain an internal fork of the linter codebase; manual formatting comments avoid the operational cost of running that custom tooling."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Delegating nits to a bot frees reviewer attention for design and correctness; handling every formatting comment manually crowds out the judgments only a human can make."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Delegating nits to a bot waives the requirement for two human approvals on a pull request; manual formatting comments preserve the dual-approval rule for any code change."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Delegating nits to a bot also silently turns off any static analysis on the same branch; manual formatting comments preserve that static analysis on every commit on the pull request."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "technical_leadership.code_review_leadership__83608",
   "topic": "technical_leadership",
   "subSkill": "code_review_leadership",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Your team uses 'LGTM' (Looks Good To Me) as an approval shorthand. You notice that LGTM approvals often miss issues that more detailed reviews catch. What change would improve this?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Require reviewers to leave at least one substantive comment (even positive) to demonstrate they actually read the code before approving"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Only allow LGTM from designated senior reviewers"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Require LGTM plus a screenshot proving the feature works"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Replace LGTM with a mandatory checklist that reviewers must complete before any approval is recorded"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "technical_leadership.code_review_leadership__83649",
   "topic": "technical_leadership",
   "subSkill": "code_review_leadership",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Your team uses 'ship/show/ask' categorization for PRs. What does 'show' mean?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Show a demo of the feature before the PR is reviewed"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Submit the PR as a draft for optional feedback"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Present the PR in standup before getting review approval"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Merge now, but open a PR so the team still sees it"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "technical_leadership.code_review_leadership__83630",
   "topic": "technical_leadership",
   "subSkill": "code_review_leadership",
   "difficulty": 2,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Your team uses PR templates with checkboxes like 'I have added tests' and 'I have updated documentation.' You notice engineers check all boxes without actually doing the work. How do you fix this?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Replace the checkboxes with automated CI checks"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Add more checkboxes for finer-grained self-certification"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Make the PR template optional since it is not working"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Require reviewers to verify each checkbox during review"
       }
      ],
      "shape": "short"
     }
    ]
   }
  },
  {
   "questionId": "technical_leadership.code_review_leadership__693282",
   "topic": "technical_leadership",
   "subSkill": "code_review_leadership",
   "difficulty": 4,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Which statement best describes how strong reviewers handle the situation when a diff exposes a deeper systemic problem?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Redirect the diff to the team that owns the systemic area and ask them to review the change instead"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Block the diff until the author also commits to fixing the systemic issue in the same pull request"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Approve the in-scope change if it is sound and file a separate ticket to address the broader issue"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Quietly approve and resolve the thread privately since reopening the issue would distract the author needlessly"
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "technical_leadership.code_review_leadership__130005",
   "topic": "technical_leadership",
   "subSkill": "code_review_leadership",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "Comparing 'trunk-based small PRs' with 'long-lived feature branches reviewed once', which framing is most useful for a staff engineer choosing a default workflow?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Trunk-based small PRs require the team to abandon any form of feature flags; long-lived branches enable feature flags as the sole way to gate incomplete features in production."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Trunk-based small PRs keep review cognitive load constant and surface conflicts early; long-lived branches concentrate review pain into one giant merge and hide drift from main."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Trunk-based small PRs cannot be paired with continuous integration in any modern platform; long-lived branches are a strict prerequisite for any kind of automated integration testing."
       }
      ],
      "shape": "long"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Trunk-based small PRs force every change to be released to customers immediately; long-lived branches allow the team to defer releases until a quarterly release-train cadence."
       }
      ],
      "shape": "long"
     }
    ]
   }
  },
  {
   "questionId": "technical_leadership.code_review_leadership__83615",
   "topic": "technical_leadership",
   "subSkill": "code_review_leadership",
   "difficulty": 3,
   "stem": [
    {
     "type": "p",
     "runs": [
      {
       "t": "text",
       "v": "You're introducing code review to a team that has never done it. They resist, saying it will slow them down. What's the most effective way to gain buy-in?"
      }
     ]
    }
   ],
   "widget": {
    "kind": "mcq",
    "options": [
     {
      "id": "o1",
      "label": [
       {
        "t": "text",
        "v": "Start with a pilot: review only high-risk PRs for 2 weeks, track issues caught, and present the data to the team"
       }
      ],
      "shape": "long"
     },
     {
      "id": "o2",
      "label": [
       {
        "t": "text",
        "v": "Mandate reviews immediately: they'll see the value once they experience it"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o3",
      "label": [
       {
        "t": "text",
        "v": "Make reviews optional and hope engineers adopt the practice voluntarily"
       }
      ],
      "shape": "short"
     },
     {
      "id": "o4",
      "label": [
       {
        "t": "text",
        "v": "Share Google's research on code review effectiveness"
       }
      ],
      "shape": "short"
     }
    ]
   }
  }
 ]
}