{"id":291,"date":"2025-12-11T00:00:08","date_gmt":"2025-12-11T00:00:08","guid":{"rendered":"https:\/\/alibkaba.com\/?p=291"},"modified":"2025-12-16T22:00:48","modified_gmt":"2025-12-16T22:00:48","slug":"how-to-engineer-your-way-out-of-questionnaire-hell","status":"publish","type":"post","link":"https:\/\/alibkaba.com\/index.php\/2025\/12\/11\/how-to-engineer-your-way-out-of-questionnaire-hell\/","title":{"rendered":"How to Engineer Your Way Out of Questionnaire Hell"},"content":{"rendered":"\n<figure class=\"wp-block-image size-large\"><img fetchpriority=\"high\" decoding=\"async\" width=\"1024\" height=\"458\" src=\"https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/chaos-1024x458.png\" alt=\"\" class=\"wp-image-295\" srcset=\"https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/chaos-1024x458.png 1024w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/chaos-300x134.png 300w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/chaos-768x344.png 768w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/chaos-1536x687.png 1536w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/chaos-1360x608.png 1360w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/chaos.png 1717w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>If you\u2019ve worked in GRC or TPRM (Third-Party Risk Management), you know the &#8220;Questionnaire Fatigue&#8221; is real.<\/p>\n\n\n\n<p>It usually looks like this: A vendor sends over a 200-question Excel sheet or a PDF that looks like it was scanned from a fax machine in 1999. A Security Analyst then spends the next three hours strictly doing CTRL+F through internal policy documents to answer questions like:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><em>Do you encrypt data at rest?<\/em>&nbsp;Yes<\/li>\n\n\n\n<li><em>Do you have an Incident Response Plan?<\/em>\u00a0Yes<\/li>\n\n\n\n<li><em>Do you require MFA?<\/em>\u00a0Yes<\/li>\n<\/ul>\n\n\n\n<p>It\u2019s necessary work, but it\u2019s not&nbsp;<em>high-value<\/em>&nbsp;work. The analyst&#8217;s brain is wasted on retrieval when it should be focused on risk analysis.<\/p>\n\n\n\n<p>I decided to engineer my way out of this problem. I didn\u2019t buy a six-figure tool; I built a custom engine using the tools we already had:&nbsp;<strong>Jira, Google Drive, and Gemini Pro<\/strong>.<\/p>\n\n\n\n<p>Here is how I automated 80% of the TPRM questionnaire workflow.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"the-architecture-serverless-glue\">The Architecture: Serverless Glue<\/h2>\n\n\n\n<p>I wanted a solution that fit naturally into the existing workflow. Work is tracked via Jira and documents (e.g. policies, etc.) are stored in Google Drive.<\/p>\n\n\n\n<p>I built the solution using&nbsp;<strong>Google Apps Script<\/strong>. It acts as the serverless &#8220;glue&#8221; connecting the systems.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1024\" height=\"463\" src=\"https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/workflow-1024x463.png\" alt=\"\" class=\"wp-image-294\" srcset=\"https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/workflow-1024x463.png 1024w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/workflow-300x136.png 300w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/workflow-768x347.png 768w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/workflow-1536x694.png 1536w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/workflow-1360x615.png 1360w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/workflow.png 1668w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p><strong>The Workflow:<\/strong><\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Trigger:<\/strong>&nbsp;A new TPRM ticket arrives in Jira<\/li>\n\n\n\n<li><strong>Ingest:<\/strong>&nbsp;The script detects the ticket, grabs the questionnaire attachment (Excel or PDF), and moves it to a processing folder in Drive<\/li>\n\n\n\n<li><strong>Context Loading (The &#8220;Smart&#8221; Part):<\/strong>&nbsp;It doesn&#8217;t just blindly feed documents to the AI, it scans the questionnaire to identify&nbsp;<em>which<\/em>&nbsp;specific documents are relevant (a &#8220;RAG-lite&#8221; approach) and loads strictly those<\/li>\n\n\n\n<li><strong>AI Drafting:<\/strong>&nbsp;It sends the context + questionnaire to&nbsp;<strong>Gemini Pro<\/strong>, enabling it to answer strictly based on the policies<\/li>\n\n\n\n<li><strong>Output:<\/strong>&nbsp;It generates a clean Google Sheet with the answers, confidence levels, and source citations, then links it back to the Jira ticket<\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"building-the-brain-technical-challenges\">Building the Brain: Technical Challenges<\/h2>\n\n\n\n<p>Writing the code (JavaScript\/Google Apps Script) was the fun part and here were the three biggest engineering hurdles I had to solve to make it reliable.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"1-solving-the-context-window-problem-with-rag-lite\">1. Solving the &#8220;Context Window&#8221; Problem with RAG-Lite<\/h3>\n\n\n\n<p>You can&#8217;t just dump 50 documents into an LLM prompt and hope for the best. It\u2019s expensive, slow, and increases hallucinations.<\/p>\n\n\n\n<p>I wrote a helper function,&nbsp;<strong>getRelevantDocuments()<\/strong>, that essentially performs a semantic triage before the main work begins.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>It extracts the text from the vendor\u2019s questionnaire<\/li>\n\n\n\n<li>It sends that text + a list of&nbsp;<em>filenames<\/em>&nbsp;of our documents to Gemini<\/li>\n\n\n\n<li>It asks:&nbsp;<em>&#8220;Based on this questionnaire, which of these documents contain the answers?&#8221;<\/em><\/li>\n\n\n\n<li>It returns a filtered list (e.g., Access_Control_Policy, SOC_2_Report, Pentest_Report, etc.)<\/li>\n<\/ol>\n\n\n\n<p>The script then opens&nbsp;<em>only<\/em>&nbsp;those files to build the context for the actual answering phase. This keeps the prompt focused and the accuracy high.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"2-taming-the-input-pdfs--excel\">2. Taming the Input (PDFs &amp; Excel)<\/h3>\n\n\n\n<p>Vendors send questionnaires in every format imaginable. While Google Apps Script is great, but it doesn&#8217;t natively parse complex PDF or Excel blobs easily.<\/p>\n\n\n\n<p>I built a parser that converts these incoming files into temporary Google Docs or Sheets on the fly to extract the raw text.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code>\/\/ Example logic in the script\nif (mimeType === MimeType.PDF) {\n    \/\/ Convert PDF to temporary Google Doc for OCR text extraction\n    const tempFile = Drive.Files.create(resource, blob, {ocr: true});\n    const text = DocumentApp.openById(tempFile.id).getBody().getText();\n    \/\/ Clean up temp file\n}<\/code><\/code><\/pre>\n\n\n\n<p>This ensures that whether the vendor sends a pristine spreadsheet or a PDF export, the engine can read it.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"3-enforcing-json-output-for-automation\">3. Enforcing JSON output for Automation<\/h3>\n\n\n\n<p>An LLM usually loves to chat.&nbsp;<em>&#8220;Here are the answers you asked for&#8230;&#8221;<\/em>, I didn&#8217;t want chat, I needed data I could push into a spreadsheet. So I used a rigorous prompt engineering to enforce a strict&nbsp;<strong>JSON array<\/strong>&nbsp;output. I instructed the model to return&nbsp;<em>only<\/em>&nbsp;a clean JSON object containing&nbsp;<strong>question<\/strong>,&nbsp;<strong>confidence<\/strong>,&nbsp;<strong>answer<\/strong>, and&nbsp;<strong>source<\/strong>.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code><code><em>\/* \nRULES FOR RESPONSE GENERATION:\n1. Every answer MUST conclude with \"(Refer to source document for details.)\"\n2. Confidence Level: 'High' vs 'Low'\n3. Source Attribution: EXACT document name required.\n*\/<\/em><\/code><\/code><\/pre>\n\n\n\n<p>I also added error handling that strips markdown code fences (<em>json<\/em>) because models love to wrap their code. This ensures the script never crashes because the AI tried to be &#8220;helpful&#8221; with formatting.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1024\" height=\"455\" src=\"https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/crumble-1024x455.png\" alt=\"\" class=\"wp-image-293\" srcset=\"https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/crumble-1024x455.png 1024w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/crumble-300x133.png 300w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/crumble-768x341.png 768w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/crumble-1536x683.png 1536w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/crumble-1360x605.png 1360w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/crumble.png 1721w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"the-result-from-days-to-minutes\">The Result: From Days to Minutes<\/h2>\n\n\n\n<p>The impact was immediate.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Speed:<\/strong>&nbsp;A questionnaire that took 3 hours to draft now takes about&nbsp;<strong>4 minutes<\/strong>&nbsp;of processing time<\/li>\n\n\n\n<li><strong>Accuracy:<\/strong>&nbsp;Because I enforced &#8220;Source Attribution,&#8221; every answer points the analyst to the exact policy file, no more guessing<\/li>\n\n\n\n<li><strong>Analyst:<\/strong>&nbsp;You now start with a draft questionnaire response that is 80-90% complete and only have to review the &#8220;Low Confidence&#8221; answers and hit send<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"591\" src=\"https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/verticals-1024x591.png\" alt=\"\" class=\"wp-image-292\" srcset=\"https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/verticals-1024x591.png 1024w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/verticals-300x173.png 300w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/verticals-768x443.png 768w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/verticals-1536x887.png 1536w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/verticals-1360x785.png 1360w, https:\/\/alibkaba.com\/wp-content\/uploads\/2025\/12\/verticals.png 1618w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"the-builder-mindset\">The &#8220;Builder&#8221; Mindset<\/h2>\n\n\n\n<p>Building this engine wasn&#8217;t just about saving time, it was a statement about how modern security teams should operate.<\/p>\n\n\n\n<p>As security leaders, we can&#8217;t just be policy writers or tool buyers, we must be builders, which means look at operational pain, understand the API capabilities of our toolset, and engineer our way to efficiency.<\/p>\n\n\n\n<p>The most valuable security tool isn&#8217;t something you buy off the shelf; it&#8217;s the ability to see a bottleneck and code your way out of it. That\u2019s how you scale security without just throwing more bodies at the problem.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>If you\u2019ve worked in GRC or TPRM (Third-Party Risk Management), you know the &#8220;Questionnaire Fatigue&#8221; is real. It usually looks&#8230;<\/p>\n","protected":false},"author":2,"featured_media":295,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-291","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blog"],"acf":[],"_links":{"self":[{"href":"https:\/\/alibkaba.com\/index.php\/wp-json\/wp\/v2\/posts\/291","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/alibkaba.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/alibkaba.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/alibkaba.com\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/alibkaba.com\/index.php\/wp-json\/wp\/v2\/comments?post=291"}],"version-history":[{"count":5,"href":"https:\/\/alibkaba.com\/index.php\/wp-json\/wp\/v2\/posts\/291\/revisions"}],"predecessor-version":[{"id":327,"href":"https:\/\/alibkaba.com\/index.php\/wp-json\/wp\/v2\/posts\/291\/revisions\/327"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/alibkaba.com\/index.php\/wp-json\/wp\/v2\/media\/295"}],"wp:attachment":[{"href":"https:\/\/alibkaba.com\/index.php\/wp-json\/wp\/v2\/media?parent=291"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/alibkaba.com\/index.php\/wp-json\/wp\/v2\/categories?post=291"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/alibkaba.com\/index.php\/wp-json\/wp\/v2\/tags?post=291"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}