Browser extensions

Thursday, 11/12 · Week 11

Post-Class Notes

TL;DR#

Today was for moving from calling a model to building a product around one, and then into the browser: what an AI product holds besides the model, what a browser extension is, and your own ideas for one. Mini Project 4 starts here.

What to do next#
  1. Mini Project 3: you will present it in class on Tuesday 11/17.
  2. The final project is on Canvas, and it must have an AI feature: a model called with instructions you wrote for your application and the person it is for. We will go through it on Tuesday 11/17. Bring your three candidate ideas to class on Thursday 11/19; the Proposal is due Friday 11/20 at 11:59 PM.
  3. Mini Project 4 is due Sunday 11/29 at 11:59 PM, after the break, and you will demo it on Tuesday 12/01. It goes in its own new public repo, with PROPOSAL.md committed before your agent writes any code.
  4. Checkpoint 6 is due Sunday 11/29. In your oim3690 repo:
    • buzzword-meter/ with one direction from Pick a Direction built on it. The steps are in Your Turn: Buzzword Meter
    • logs/wk11.md with your three ideas under ## MP4 Wish List, and the one you chose for Mini Project 4 marked
Where your instructions show up#

The instructions in api/ask.js are added on the server, and they still reach the browser: the model's reply includes them, and your function passes the whole reply to your page. Open Network, click the ask request and read Response. A visitor can do the same, so keep anything private out of your instructions.

A chat that remembers#

The API keeps nothing between calls. A chat that remembers sends every message so far in each request, so the body of the next request holds every earlier question and answer plus the new question. Keeping the last request body is enough, because it already holds the whole conversation. Each turn sends more than the last and costs more, which is why a new task belongs in a new conversation, with your agent too.

Retrieval augmented generation#

A model answers about material it has never seen only when that material is in the request. Pasting the whole syllabus works for one short document. A product with a lot of material, such as a shop's policies and product pages, searches it first and sends only the passages that match the question. That search step is what retrieval augmented generation (RAG) names. Two things hold either way: every passage you send is paid for on every request, and it goes to the model provider like the rest of the request.

Different answers and wrong answers#

The same question can get different answers because the model picks each word with some randomness. A confident answer that is false is a separate problem, and it happens without randomness too. Check an answer against its source whenever it matters.

How a product uses a model#

From the last slides of Calling an LLM API, from the simplest use to the most involved:

  • Show an answer: the visitor types, the model replies, the page shows it.
  • Answer from your data: send your own material with the question.
  • Act on the answer: ask for JSON, such as {"style": "vegetarian", "max_price": 15}, and let your code do the work with it.
  • Add a tool: let the model ask your code to run a function, such as one that returns today's date.

For Mini Project 4 and your final project, decide which of these your idea needs, and what your page does with the answer.

Seeing what your agent sends#

To see what one question to Copilot costs, press Ctrl+Shift+P (Windows) or Cmd+Shift+P (macOS), run Developer: Show Chat Debug View, and ask a question in a new chat. The log shows the instructions Copilot sends every time, each tool call it makes, such as reading your files, and the tokens used. In class one short question about ai-chat.html used about 83,000 tokens.

Keeping a key out of your agent's commands#

When you approve a command with "Yes, and don't ask again", Claude Code saves that command to its settings file, key included. Apiiro found about 1,100 of those files with secrets in public GitHub repos (Ars Technica). Paste a key into .env or Vercel's settings page, and keep it out of every command and every chat message you give your agent.

Starting an extension#
  • An extension is a folder: manifest.json, a script and sometimes CSS. To install yours, open chrome://extensions, turn on Developer mode, click Load unpacked and pick the folder.
  • After every change, reload the extension on chrome://extensions, then refresh the page. A fix that seems to do nothing is usually a missing reload.
  • Check the manifest before anything else. Chrome runs only Manifest V3, and many older tutorials are V2. Check the Manifest First lists what to look for.
  • What an Extension Cannot Do is worth reading before you pick an idea. Two of its points, put more precisely than in class: your script runs apart from the page's own JavaScript, and "world": "MAIN" in the manifest's content_scripts entry runs it inside; and Chrome stops the background script after 30 seconds with nothing to do, so anything that has to last goes in chrome.storage.
Finding an idea for Mini Project 4#

Open chrome://history, look at the last week, and pick one of the sites you open most. Find something you do there by hand every time, and write it as I wish my browser would ___ when I am on ___. The second blank becomes matches in your manifest.

Generative engine optimization#

After the lightning talk the discussion turned to how products get into a chatbot's answers. Generative engine optimization (GEO) is the name for shaping web content so that generative search engines cite it, by analogy with SEO. The term comes from a 2023 paper, GEO: Generative Engine Optimization. One tactic that came up, publishing thousands of pages that each answer one narrow question, is what Google's spam policies call scaled content abuse. Both are worth reading if your project depends on being found.