PlanOpticon

docs: comprehensive v0.4.0 documentation — 27 pages, use cases, FAQ New pages (11): - guide/companion.md — Interactive Companion REPL - guide/planning-agent.md — Planning Agent and 11 skills - guide/knowledge-graphs.md — KG storage, querying, taxonomy, viewer - guide/authentication.md — OAuth setup for 6 services - guide/document-ingestion.md — PDF, Markdown, plaintext ingestion - guide/export.md — 7 markdown doc types, Obsidian, Notion, Wiki, Exchange - api/agent.md — PlanningAgent, AgentContext, Skills API - api/sources.md — BaseSource, 21 source connectors - api/auth.md — AuthConfig, OAuthManager API - use-cases.md — 10 real-world workflows with full commands - faq.md — FAQ and troubleshooting guide Updated pages (10): - guide/output-formats.md — all output formats including SQLite KG, Exchange - guide/single-video.md — taxonomy, --speakers, --output-format, post-analysis - guide/batch.md — fuzzy merge, querying results, incremental processing - architecture/pipeline.md — 5 mermaid diagrams, all pipelines - contributing.md — ruff, ProviderRegistry, skills, processors, exporters - getting-started/configuration.md — full .env example with OAuth walkthroughs - api/models.md — all 17+ Pydantic models documented - api/providers.md — BaseProvider, ProviderRegistry, ProviderManager - api/analyzers.md — DiagramAnalyzer, ContentAnalyzer, ActionDetector - mkdocs.yml — nav updated with all new pages Also fixes check-yaml pre-commit hook to handle mkdocs.yml Python tags.

lmata 2026-03-08 00:17 UTC trunk

Commit 3da1f8f9af3d2ae023942b141853a68b784a7fd986e8a1e92f650e182fe78dbd

Parent 0981a082d9bd09a…

22 files changed +1 +407 +382 +377 +498 +496 +281 +292 -14 +444 -13 +301 +141 -25 +525 +120 -10 +531 +434 +756 +650 +303 -21 +425 +190 -14 +342 +11

~ .pre-commit-config.yaml + docs/api/agent.md ~ docs/api/analyzers.md + docs/api/auth.md ~ docs/api/models.md ~ docs/api/providers.md + docs/api/sources.md ~ docs/architecture/pipeline.md ~ docs/contributing.md + docs/faq.md ~ docs/getting-started/configuration.md + docs/guide/authentication.md ~ docs/guide/batch.md + docs/guide/companion.md + docs/guide/document-ingestion.md + docs/guide/export.md + docs/guide/knowledge-graphs.md ~ docs/guide/output-formats.md + docs/guide/planning-agent.md ~ docs/guide/single-video.md + docs/use-cases.md ~ mkdocs.yml

M .pre-commit-config.yaml

		--- .pre-commit-config.yaml
		+++ .pre-commit-config.yaml
		@@ -9,9 +9,10 @@
9	9	rev: v5.0.0
10	10	hooks:
11	11	- id: trailing-whitespace
12	12	- id: end-of-file-fixer
13	13	- id: check-yaml
	14	+ args: [--unsafe]
14	15	- id: check-added-large-files
15	16	args: [--maxkb=500]
16	17	- id: check-merge-conflict
17	18	- id: detect-private-key
18	19
19	20	ADDED docs/api/agent.md

	--- .pre-commit-config.yaml
	+++ .pre-commit-config.yaml
	@@ -9,9 +9,10 @@
9	rev: v5.0.0
10	hooks:
11	- id: trailing-whitespace
12	- id: end-of-file-fixer
13	- id: check-yaml

14	- id: check-added-large-files
15	args: [--maxkb=500]
16	- id: check-merge-conflict
17	- id: detect-private-key
18
19	DDED docs/api/agent.md

	--- .pre-commit-config.yaml
	+++ .pre-commit-config.yaml
	@@ -9,9 +9,10 @@
9	rev: v5.0.0
10	hooks:
11	- id: trailing-whitespace
12	- id: end-of-file-fixer
13	- id: check-yaml
14	args: [--unsafe]
15	- id: check-added-large-files
16	args: [--maxkb=500]
17	- id: check-merge-conflict
18	- id: detect-private-key
19
20	DDED docs/api/agent.md

A docs/api/agent.md

+407

		--- a/docs/api/agent.md
		+++ b/docs/api/agent.md
		@@ -0,0 +1,407 @@
	1	+# Agent API Reference
	2	+
	3	+::: video_processor.agent.agent_loop
	4	+
	5	+::: video_processor.agent.skills.base
	6	+
	7	+::: video_processor.agent.kb_context
	8	+
	9	+---
	10	+
	11	+## Overview
	12	+
	13	+The agent module implements a planning agent that synthesizes knowledge from processed video content into actionable artifacts such as project plans, PRDs, task breakdowns, and roadmaps. The agent operates on knowledge graphs loaded via `KBContext` and uses a skill-based architecture for extensibility.
	14	+
	15	+Key components:
	16	+
	17	+- `PlanningAgent` -- orchestrates skill selection and execution based on user requests
	18	+- `AgentContext` -- shared state passed between skills during execution
	19	+- `Skill` (ABC) -- base class for pluggable agent capabilities
	20	+- `Artifact` -- output produced by skill execution
	21	+- `KBContext` -- loads and merges multiple knowledge graph sources
	22	+
	23	+---
	24	+
	25	+## PlanningAgent
	26	+
	27	+```python
	28	+from video_processor.agent.agent_loop import PlanningAgent
	29	+```
	30	+
	31	+AI agent that synthesizes knowledge into planning artifacts. Uses an LLM to select which skills to execute for a given request, or falls back to keyword matching when no LLM is available.
	32	+
	33	+### Constructor
	34	+
	35	+```python
	36	+def __init__(self, context: AgentContext)
	37	+```
	38	+
	39	+\| Parameter \| Type \| Description \|
	40	+\|---\|---\|---\|
	41	+\| `context` \| `AgentContext` \| Shared context containing knowledge graph, query engine, and provider \|
	42	+
	43	+### from_kb_paths()
	44	+
	45	+```python
	46	+@classmethod
	47	+def from_kb_paths(
	48	+ cls,
	49	+ kb_paths: List[Path],
	50	+ provider_manager=None,
	51	+) -> PlanningAgent
	52	+```
	53	+
	54	+Factory method that creates an agent from one or more knowledge base file paths. Handles loading and merging knowledge graphs automatically.
	55	+
	56	+Parameters:
	57	+
	58	+\| Parameter \| Type \| Default \| Description \|
	59	+\|---\|---\|---\|---\|
	60	+\| `kb_paths` \| `List[Path]` \| required \| Paths to `.db` or `.json` knowledge graph files, or directories to search \|
	61	+\| `provider_manager` \| `ProviderManager` \| `None` \| LLM provider for agent operations \|
	62	+
	63	+Returns: `PlanningAgent` -- configured agent with loaded knowledge base.
	64	+
	65	+```python
	66	+from pathlib import Path
	67	+from video_processor.agent.agent_loop import PlanningAgent
	68	+from video_processor.providers.manager import ProviderManager
	69	+
	70	+agent = PlanningAgent.from_kb_paths(
	71	+ kb_paths=[Path("results/knowledge_graph.db")],
	72	+ provider_manager=ProviderManager(),
	73	+)
	74	+```
	75	+
	76	+### execute()
	77	+
	78	+```python
	79	+def execute(self, request: str) -> List[Artifact]
	80	+```
	81	+
	82	+Execute a user request by selecting and running appropriate skills.
	83	+
	84	+Process:
	85	+
	86	+1. Build a context summary from the knowledge base statistics
	87	+2. Format available skills with their descriptions
	88	+3. Ask the LLM to select skills and parameters (or use keyword matching as fallback)
	89	+4. Execute selected skills in order, accumulating artifacts
	90	+
	91	+Parameters:
	92	+
	93	+\| Parameter \| Type \| Description \|
	94	+\|---\|---\|---\|
	95	+\| `request` \| `str` \| Natural language request (e.g., "Generate a project plan") \|
	96	+
	97	+Returns: `List[Artifact]` -- generated artifacts from skill execution.
	98	+
	99	+LLM mode: The LLM receives the knowledge base summary, available skills, and user request, then returns a JSON array of `{"skill": "name", "params": {}}` objects to execute.
	100	+
	101	+Keyword fallback: Without an LLM, skills are matched by splitting the skill name into words and checking if any appear in the request text.
	102	+
	103	+```python
	104	+artifacts = agent.execute("Create a PRD and task breakdown")
	105	+for artifact in artifacts:
	106	+ print(f"--- {artifact.name} ({artifact.artifact_type}) ---")
	107	+ print(artifact.content[:500])
	108	+```
	109	+
	110	+### chat()
	111	+
	112	+```python
	113	+def chat(self, message: str) -> str
	114	+```
	115	+
	116	+Interactive chat mode. Maintains conversation history and provides contextual responses about the loaded knowledge base.
	117	+
	118	+Parameters:
	119	+
	120	+\| Parameter \| Type \| Description \|
	121	+\|---\|---\|---\|
	122	+\| `message` \| `str` \| User message \|
	123	+
	124	+Returns: `str` -- assistant response.
	125	+
	126	+The chat mode provides the LLM with:
	127	+
	128	+- Knowledge base statistics (entity counts, relationship counts)
	129	+- List of previously generated artifacts
	130	+- Full conversation history
	131	+- Available REPL commands (e.g., `/entities`, `/search`, `/plan`, `/export`)
	132	+
	133	+Requires a configured `provider_manager`. Returns a static error message if no LLM is available.
	134	+
	135	+```python
	136	+response = agent.chat("What technologies were discussed in the meetings?")
	137	+print(response)
	138	+
	139	+response = agent.chat("Which of those have the most dependencies?")
	140	+print(response)
	141	+```
	142	+
	143	+---
	144	+
	145	+## AgentContext
	146	+
	147	+```python
	148	+from video_processor.agent.skills.base import AgentContext
	149	+```
	150	+
	151	+Shared state dataclass passed to all skills during execution. Accumulates artifacts and conversation history across the agent session.
	152	+
	153	+\| Field \| Type \| Default \| Description \|
	154	+\|---\|---\|---\|---\|
	155	+\| `knowledge_graph` \| `Any` \| `None` \| `KnowledgeGraph` instance \|
	156	+\| `query_engine` \| `Any` \| `None` \| `GraphQueryEngine` instance for querying the KG \|
	157	+\| `provider_manager` \| `Any` \| `None` \| `ProviderManager` instance for LLM calls \|
	158	+\| `planning_entities` \| `List[Any]` \| `[]` \| Extracted `PlanningEntity` instances \|
	159	+\| `user_requirements` \| `Dict[str, Any]` \| `{}` \| User-specified requirements and constraints \|
	160	+\| `conversation_history` \| `List[Dict[str, str]]` \| `[]` \| Chat message history (`role`, `content` dicts) \|
	161	+\| `artifacts` \| `List[Artifact]` \| `[]` \| Previously generated artifacts \|
	162	+\| `config` \| `Dict[str, Any]` \| `{}` \| Additional configuration \|
	163	+
	164	+```python
	165	+from video_processor.agent.skills.base import AgentContext
	166	+
	167	+context = AgentContext(
	168	+ knowledge_graph=kg,
	169	+ query_engine=engine,
	170	+ provider_manager=pm,
	171	+ config={"output_format": "markdown"},
	172	+)
	173	+```
	174	+
	175	+---
	176	+
	177	+## Skill (ABC)
	178	+
	179	+```python
	180	+from video_processor.agent.skills.base import Skill
	181	+```
	182	+
	183	+Base class for agent skills. Each skill represents a discrete capability that produces an artifact from the agent context.
	184	+
	185	+Class attributes:
	186	+
	187	+\| Attribute \| Type \| Description \|
	188	+\|---\|---\|---\|
	189	+\| `name` \| `str` \| Skill identifier (e.g., `"project_plan"`, `"prd"`) \|
	190	+\| `description` \| `str` \| Human-readable description shown to the LLM for skill selection \|
	191	+
	192	+### execute()
	193	+
	194	+```python
	195	+@abstractmethod
	196	+def execute(self, context: AgentContext, **kwargs) -> Artifact
	197	+```
	198	+
	199	+Execute this skill and return an artifact. Receives the shared agent context and any parameters selected by the LLM planner.
	200	+
	201	+### can_execute()
	202	+
	203	+```python
	204	+def can_execute(self, context: AgentContext) -> bool
	205	+```
	206	+
	207	+Check if this skill can execute given the current context. The default implementation requires both `knowledge_graph` and `provider_manager` to be set. Override for skills with different requirements.
	208	+
	209	+Returns: `bool`
	210	+
	211	+### Implementing a custom skill
	212	+
	213	+```python
	214	+from video_processor.agent.skills.base import Skill, Artifact, AgentContext, register_skill
	215	+
	216	+class SummarySkill(Skill):
	217	+ name = "summary"
	218	+ description = "Generate a concise summary of the knowledge base"
	219	+
	220	+ def execute(self, context: AgentContext, **kwargs) -> Artifact:
	221	+ stats = context.query_engine.stats()
	222	+ prompt = f"Summarize this knowledge base:\n{stats.to_text()}"
	223	+ content = context.provider_manager.chat(
	224	+ [{"role": "user", "content": prompt}]
	225	+ )
	226	+ return Artifact(
	227	+ name="Knowledge Base Summary",
	228	+ content=content,
	229	+ artifact_type="document",
	230	+ format="markdown",
	231	+ )
	232	+
	233	+ def can_execute(self, context: AgentContext) -> bool:
	234	+ return context.query_engine is not None and context.provider_manager is not None
	235	+
	236	+# Register the skill so the agent can discover it
	237	+register_skill(SummarySkill())
	238	+```
	239	+
	240	+---
	241	+
	242	+## Artifact
	243	+
	244	+```python
	245	+from video_processor.agent.skills.base import Artifact
	246	+```
	247	+
	248	+Dataclass representing the output of a skill execution.
	249	+
	250	+\| Field \| Type \| Default \| Description \|
	251	+\|---\|---\|---\|---\|
	252	+\| `name` \| `str` \| required \| Human-readable artifact name \|
	253	+\| `content` \| `str` \| required \| Generated content (Markdown, JSON, Mermaid, etc.) \|
	254	+\| `artifact_type` \| `str` \| required \| Type: `"project_plan"`, `"prd"`, `"roadmap"`, `"task_list"`, `"document"`, `"issues"` \|
	255	+\| `format` \| `str` \| `"markdown"` \| Content format: `"markdown"`, `"json"`, `"mermaid"` \|
	256	+\| `metadata` \| `Dict[str, Any]` \| `{}` \| Additional metadata \|
	257	+
	258	+---
	259	+
	260	+## Skill Registry Functions
	261	+
	262	+### register_skill()
	263	+
	264	+```python
	265	+def register_skill(skill: Skill) -> None
	266	+```
	267	+
	268	+Register a skill instance in the global registry. Skills must be registered before the agent can discover and execute them.
	269	+
	270	+### get_skill()
	271	+
	272	+```python
	273	+def get_skill(name: str) -> Optional[Skill]
	274	+```
	275	+
	276	+Look up a registered skill by name.
	277	+
	278	+Returns: `Optional[Skill]` -- the skill instance, or `None` if not found.
	279	+
	280	+### list_skills()
	281	+
	282	+```python
	283	+def list_skills() -> List[Skill]
	284	+```
	285	+
	286	+Return all registered skill instances.
	287	+
	288	+---
	289	+
	290	+## KBContext
	291	+
	292	+```python
	293	+from video_processor.agent.kb_context import KBContext
	294	+```
	295	+
	296	+Loads and merges multiple knowledge graph sources into a unified context for agent consumption. Supports both FalkorDB (`.db`) and JSON (`.json`) formats, and can auto-discover graphs in a directory tree.
	297	+
	298	+### Constructor
	299	+
	300	+```python
	301	+def __init__(self)
	302	+```
	303	+
	304	+Creates an empty context. Use `add_source()` to add knowledge graph paths, then `load()` to initialize.
	305	+
	306	+### add_source()
	307	+
	308	+```python
	309	+def add_source(self, path) -> None
	310	+```
	311	+
	312	+Add a knowledge graph source.
	313	+
	314	+Parameters:
	315	+
	316	+\| Parameter \| Type \| Description \|
	317	+\|---\|---\|---\|
	318	+\| `path` \| `str \\| Path` \| Path to a `.db` file, `.json` file, or directory to search for knowledge graphs \|
	319	+
	320	+If `path` is a directory, it is searched recursively for knowledge graph files using `find_knowledge_graphs()`.
	321	+
	322	+Raises: `FileNotFoundError` if the path does not exist.
	323	+
	324	+### load()
	325	+
	326	+```python
	327	+def load(self, provider_manager=None) -> KBContext
	328	+```
	329	+
	330	+Load and merge all added sources into a single knowledge graph and query engine.
	331	+
	332	+Parameters:
	333	+
	334	+\| Parameter \| Type \| Default \| Description \|
	335	+\|---\|---\|---\|---\|
	336	+\| `provider_manager` \| `ProviderManager` \| `None` \| LLM provider for the knowledge graph and query engine \|
	337	+
	338	+Returns: `KBContext` -- self, for method chaining.
	339	+
	340	+### Properties
	341	+
	342	+\| Property \| Type \| Description \|
	343	+\|---\|---\|---\|
	344	+\| `knowledge_graph` \| `KnowledgeGraph` \| The merged knowledge graph (raises `RuntimeError` if not loaded) \|
	345	+\| `query_engine` \| `GraphQueryEngine` \| Query engine for the merged graph (raises `RuntimeError` if not loaded) \|
	346	+\| `sources` \| `List[Path]` \| List of resolved source paths \|
	347	+
	348	+### summary()
	349	+
	350	+```python
	351	+def summary(self) -> str
	352	+```
	353	+
	354	+Generate a brief text summary of the loaded knowledge base, including entity counts by type and relationship counts.
	355	+
	356	+Returns: `str` -- multi-line summary text.
	357	+
	358	+### auto_discover()
	359	+
	360	+```python
	361	+@classmethod
	362	+def auto_discover(
	363	+ cls,
	364	+ start_dir: Optional[Path] = None,
	365	+ provider_manager=None,
	366	+) -> KBContext
	367	+```
	368	+
	369	+Factory method that creates a `KBContext` by auto-discovering knowledge graphs near `start_dir` (defaults to current directory).
	370	+
	371	+Returns: `KBContext` -- loaded context (may have zero sources if none found).
	372	+
	373	+### Usage examples
	374	+
	375	+```python
	376	+from pathlib import Path
	377	+from video_processor.agent.kb_context import KBContext
	378	+
	379	+# Manual source management
	380	+kb = KBContext()
	381	+kb.add_source(Path("project_a/knowledge_graph.db"))
	382	+kb.add_source(Path("project_b/results/")) # searches directory
	383	+kb.load(provider_manager=pm)
	384	+
	385	+print(kb.summary())
	386	+# Knowledge base: 3 source(s)
	387	+# Entities: 142
	388	+# Relationships: 89
	389	+# Entity types:
	390	+# technology: 45
	391	+# person: 23
	392	+# concept: 74
	393	+
	394	+# Auto-discover from current directory
	395	+kb = KBContext.auto_discover()
	396	+
	397	+# Use with the agent
	398	+from video_processor.agent.agent_loop import PlanningAgent
	399	+from video_processor.agent.skills.base import AgentContext
	400	+
	401	+context = AgentContext(
	402	+ knowledge_graph=kb.knowledge_graph,
	403	+ query_engine=kb.query_engine,
	404	+ provider_manager=pm,
	405	+)
	406	+agent = PlanningAgent(context)
	407	+```

	--- a/docs/api/agent.md
	+++ b/docs/api/agent.md
	@@ -0,0 +1,407 @@

	--- a/docs/api/agent.md
	+++ b/docs/api/agent.md
	@@ -0,0 +1,407 @@
1	# Agent API Reference
2
3	::: video_processor.agent.agent_loop
4
5	::: video_processor.agent.skills.base
6
7	::: video_processor.agent.kb_context
8
9	---
10
11	## Overview
12
13	The agent module implements a planning agent that synthesizes knowledge from processed video content into actionable artifacts such as project plans, PRDs, task breakdowns, and roadmaps. The agent operates on knowledge graphs loaded via `KBContext` and uses a skill-based architecture for extensibility.
14
15	Key components:
16
17	- `PlanningAgent` -- orchestrates skill selection and execution based on user requests
18	- `AgentContext` -- shared state passed between skills during execution
19	- `Skill` (ABC) -- base class for pluggable agent capabilities
20	- `Artifact` -- output produced by skill execution
21	- `KBContext` -- loads and merges multiple knowledge graph sources
22
23	---
24
25	## PlanningAgent
26
27	```python
28	from video_processor.agent.agent_loop import PlanningAgent
29	```
30
31	AI agent that synthesizes knowledge into planning artifacts. Uses an LLM to select which skills to execute for a given request, or falls back to keyword matching when no LLM is available.
32
33	### Constructor
34
35	```python
36	def __init__(self, context: AgentContext)
37	```
38
39	\| Parameter \| Type \| Description \|
40	\|---\|---\|---\|
41	\| `context` \| `AgentContext` \| Shared context containing knowledge graph, query engine, and provider \|
42
43	### from_kb_paths()
44
45	```python
46	@classmethod
47	def from_kb_paths(
48	cls,
49	kb_paths: List[Path],
50	provider_manager=None,
51	) -> PlanningAgent
52	```
53
54	Factory method that creates an agent from one or more knowledge base file paths. Handles loading and merging knowledge graphs automatically.
55
56	Parameters:
57
58	\| Parameter \| Type \| Default \| Description \|
59	\|---\|---\|---\|---\|
60	\| `kb_paths` \| `List[Path]` \| required \| Paths to `.db` or `.json` knowledge graph files, or directories to search \|
61	\| `provider_manager` \| `ProviderManager` \| `None` \| LLM provider for agent operations \|
62
63	Returns: `PlanningAgent` -- configured agent with loaded knowledge base.
64
65	```python
66	from pathlib import Path
67	from video_processor.agent.agent_loop import PlanningAgent
68	from video_processor.providers.manager import ProviderManager
69
70	agent = PlanningAgent.from_kb_paths(
71	kb_paths=[Path("results/knowledge_graph.db")],
72	provider_manager=ProviderManager(),
73	)
74	```
75
76	### execute()
77
78	```python
79	def execute(self, request: str) -> List[Artifact]
80	```
81
82	Execute a user request by selecting and running appropriate skills.
83
84	Process:
85
86	1. Build a context summary from the knowledge base statistics
87	2. Format available skills with their descriptions
88	3. Ask the LLM to select skills and parameters (or use keyword matching as fallback)
89	4. Execute selected skills in order, accumulating artifacts
90
91	Parameters:
92
93	\| Parameter \| Type \| Description \|
94	\|---\|---\|---\|
95	\| `request` \| `str` \| Natural language request (e.g., "Generate a project plan") \|
96
97	Returns: `List[Artifact]` -- generated artifacts from skill execution.
98
99	LLM mode: The LLM receives the knowledge base summary, available skills, and user request, then returns a JSON array of `{"skill": "name", "params": {}}` objects to execute.
100
101	Keyword fallback: Without an LLM, skills are matched by splitting the skill name into words and checking if any appear in the request text.
102
103	```python
104	artifacts = agent.execute("Create a PRD and task breakdown")
105	for artifact in artifacts:
106	print(f"--- {artifact.name} ({artifact.artifact_type}) ---")
107	print(artifact.content[:500])
108	```
109
110	### chat()
111
112	```python
113	def chat(self, message: str) -> str
114	```
115
116	Interactive chat mode. Maintains conversation history and provides contextual responses about the loaded knowledge base.
117
118	Parameters:
119
120	\| Parameter \| Type \| Description \|
121	\|---\|---\|---\|
122	\| `message` \| `str` \| User message \|
123
124	Returns: `str` -- assistant response.
125
126	The chat mode provides the LLM with:
127
128	- Knowledge base statistics (entity counts, relationship counts)
129	- List of previously generated artifacts
130	- Full conversation history
131	- Available REPL commands (e.g., `/entities`, `/search`, `/plan`, `/export`)
132
133	Requires a configured `provider_manager`. Returns a static error message if no LLM is available.
134
135	```python
136	response = agent.chat("What technologies were discussed in the meetings?")
137	print(response)
138
139	response = agent.chat("Which of those have the most dependencies?")
140	print(response)
141	```
142
143	---
144
145	## AgentContext
146
147	```python
148	from video_processor.agent.skills.base import AgentContext
149	```
150
151	Shared state dataclass passed to all skills during execution. Accumulates artifacts and conversation history across the agent session.
152
153	\| Field \| Type \| Default \| Description \|
154	\|---\|---\|---\|---\|
155	\| `knowledge_graph` \| `Any` \| `None` \| `KnowledgeGraph` instance \|
156	\| `query_engine` \| `Any` \| `None` \| `GraphQueryEngine` instance for querying the KG \|
157	\| `provider_manager` \| `Any` \| `None` \| `ProviderManager` instance for LLM calls \|
158	\| `planning_entities` \| `List[Any]` \| `[]` \| Extracted `PlanningEntity` instances \|
159	\| `user_requirements` \| `Dict[str, Any]` \| `{}` \| User-specified requirements and constraints \|
160	\| `conversation_history` \| `List[Dict[str, str]]` \| `[]` \| Chat message history (`role`, `content` dicts) \|
161	\| `artifacts` \| `List[Artifact]` \| `[]` \| Previously generated artifacts \|
162	\| `config` \| `Dict[str, Any]` \| `{}` \| Additional configuration \|
163
164	```python
165	from video_processor.agent.skills.base import AgentContext
166
167	context = AgentContext(
168	knowledge_graph=kg,
169	query_engine=engine,
170	provider_manager=pm,
171	config={"output_format": "markdown"},
172	)
173	```
174
175	---
176
177	## Skill (ABC)
178
179	```python
180	from video_processor.agent.skills.base import Skill
181	```
182
183	Base class for agent skills. Each skill represents a discrete capability that produces an artifact from the agent context.
184
185	Class attributes:
186
187	\| Attribute \| Type \| Description \|
188	\|---\|---\|---\|
189	\| `name` \| `str` \| Skill identifier (e.g., `"project_plan"`, `"prd"`) \|
190	\| `description` \| `str` \| Human-readable description shown to the LLM for skill selection \|
191
192	### execute()
193
194	```python
195	@abstractmethod
196	def execute(self, context: AgentContext, **kwargs) -> Artifact
197	```
198
199	Execute this skill and return an artifact. Receives the shared agent context and any parameters selected by the LLM planner.
200
201	### can_execute()
202
203	```python
204	def can_execute(self, context: AgentContext) -> bool
205	```
206
207	Check if this skill can execute given the current context. The default implementation requires both `knowledge_graph` and `provider_manager` to be set. Override for skills with different requirements.
208
209	Returns: `bool`
210
211	### Implementing a custom skill
212
213	```python
214	from video_processor.agent.skills.base import Skill, Artifact, AgentContext, register_skill
215
216	class SummarySkill(Skill):
217	name = "summary"
218	description = "Generate a concise summary of the knowledge base"
219
220	def execute(self, context: AgentContext, **kwargs) -> Artifact:
221	stats = context.query_engine.stats()
222	prompt = f"Summarize this knowledge base:\n{stats.to_text()}"
223	content = context.provider_manager.chat(
224	[{"role": "user", "content": prompt}]
225	)
226	return Artifact(
227	name="Knowledge Base Summary",
228	content=content,
229	artifact_type="document",
230	format="markdown",
231	)
232
233	def can_execute(self, context: AgentContext) -> bool:
234	return context.query_engine is not None and context.provider_manager is not None
235
236	# Register the skill so the agent can discover it
237	register_skill(SummarySkill())
238	```
239
240	---
241
242	## Artifact
243
244	```python
245	from video_processor.agent.skills.base import Artifact
246	```
247
248	Dataclass representing the output of a skill execution.
249
250	\| Field \| Type \| Default \| Description \|
251	\|---\|---\|---\|---\|
252	\| `name` \| `str` \| required \| Human-readable artifact name \|
253	\| `content` \| `str` \| required \| Generated content (Markdown, JSON, Mermaid, etc.) \|
254	\| `artifact_type` \| `str` \| required \| Type: `"project_plan"`, `"prd"`, `"roadmap"`, `"task_list"`, `"document"`, `"issues"` \|
255	\| `format` \| `str` \| `"markdown"` \| Content format: `"markdown"`, `"json"`, `"mermaid"` \|
256	\| `metadata` \| `Dict[str, Any]` \| `{}` \| Additional metadata \|
257
258	---
259
260	## Skill Registry Functions
261
262	### register_skill()
263
264	```python
265	def register_skill(skill: Skill) -> None
266	```
267
268	Register a skill instance in the global registry. Skills must be registered before the agent can discover and execute them.
269
270	### get_skill()
271
272	```python
273	def get_skill(name: str) -> Optional[Skill]
274	```
275
276	Look up a registered skill by name.
277
278	Returns: `Optional[Skill]` -- the skill instance, or `None` if not found.
279
280	### list_skills()
281
282	```python
283	def list_skills() -> List[Skill]
284	```
285
286	Return all registered skill instances.
287
288	---
289
290	## KBContext
291
292	```python
293	from video_processor.agent.kb_context import KBContext
294	```
295
296	Loads and merges multiple knowledge graph sources into a unified context for agent consumption. Supports both FalkorDB (`.db`) and JSON (`.json`) formats, and can auto-discover graphs in a directory tree.
297
298	### Constructor
299
300	```python
301	def __init__(self)
302	```
303
304	Creates an empty context. Use `add_source()` to add knowledge graph paths, then `load()` to initialize.
305
306	### add_source()
307
308	```python
309	def add_source(self, path) -> None
310	```
311
312	Add a knowledge graph source.
313
314	Parameters:
315
316	\| Parameter \| Type \| Description \|
317	\|---\|---\|---\|
318	\| `path` \| `str \\| Path` \| Path to a `.db` file, `.json` file, or directory to search for knowledge graphs \|
319
320	If `path` is a directory, it is searched recursively for knowledge graph files using `find_knowledge_graphs()`.
321
322	Raises: `FileNotFoundError` if the path does not exist.
323
324	### load()
325
326	```python
327	def load(self, provider_manager=None) -> KBContext
328	```
329
330	Load and merge all added sources into a single knowledge graph and query engine.
331
332	Parameters:
333
334	\| Parameter \| Type \| Default \| Description \|
335	\|---\|---\|---\|---\|
336	\| `provider_manager` \| `ProviderManager` \| `None` \| LLM provider for the knowledge graph and query engine \|
337
338	Returns: `KBContext` -- self, for method chaining.
339
340	### Properties
341
342	\| Property \| Type \| Description \|
343	\|---\|---\|---\|
344	\| `knowledge_graph` \| `KnowledgeGraph` \| The merged knowledge graph (raises `RuntimeError` if not loaded) \|
345	\| `query_engine` \| `GraphQueryEngine` \| Query engine for the merged graph (raises `RuntimeError` if not loaded) \|
346	\| `sources` \| `List[Path]` \| List of resolved source paths \|
347
348	### summary()
349
350	```python
351	def summary(self) -> str
352	```
353
354	Generate a brief text summary of the loaded knowledge base, including entity counts by type and relationship counts.
355
356	Returns: `str` -- multi-line summary text.
357
358	### auto_discover()
359
360	```python
361	@classmethod
362	def auto_discover(
363	cls,
364	start_dir: Optional[Path] = None,
365	provider_manager=None,
366	) -> KBContext
367	```
368
369	Factory method that creates a `KBContext` by auto-discovering knowledge graphs near `start_dir` (defaults to current directory).
370
371	Returns: `KBContext` -- loaded context (may have zero sources if none found).
372
373	### Usage examples
374
375	```python
376	from pathlib import Path
377	from video_processor.agent.kb_context import KBContext
378
379	# Manual source management
380	kb = KBContext()
381	kb.add_source(Path("project_a/knowledge_graph.db"))
382	kb.add_source(Path("project_b/results/")) # searches directory
383	kb.load(provider_manager=pm)
384
385	print(kb.summary())
386	# Knowledge base: 3 source(s)
387	# Entities: 142
388	# Relationships: 89
389	# Entity types:
390	# technology: 45
391	# person: 23
392	# concept: 74
393
394	# Auto-discover from current directory
395	kb = KBContext.auto_discover()
396
397	# Use with the agent
398	from video_processor.agent.agent_loop import PlanningAgent
399	from video_processor.agent.skills.base import AgentContext
400
401	context = AgentContext(
402	knowledge_graph=kb.knowledge_graph,
403	query_engine=kb.query_engine,
404	provider_manager=pm,
405	)
406	agent = PlanningAgent(context)
407	```

M docs/api/analyzers.md

+382

		--- docs/api/analyzers.md
		+++ docs/api/analyzers.md
		@@ -3,5 +3,387 @@
3	3	::: video_processor.analyzers.diagram_analyzer
4	4
5	5	::: video_processor.analyzers.content_analyzer
6	6
7	7	::: video_processor.analyzers.action_detector
	8	+
	9	+---
	10	+
	11	+## Overview
	12	+
	13	+The analyzers module contains the core content extraction logic for PlanOpticon. These analyzers process video frames and transcripts to extract structured knowledge: diagrams, key points, action items, and cross-referenced entities.
	14	+
	15	+All analyzers accept an optional `ProviderManager` instance. When provided, they use LLM capabilities for richer extraction. Without one, they fall back to heuristic/pattern-based methods where possible.
	16	+
	17	+---
	18	+
	19	+## DiagramAnalyzer
	20	+
	21	+```python
	22	+from video_processor.analyzers.diagram_analyzer import DiagramAnalyzer
	23	+```
	24	+
	25	+Vision model-based diagram detection and analysis. Classifies video frames as diagrams, slides, screenshots, or other content, then performs full extraction on high-confidence frames.
	26	+
	27	+### Constructor
	28	+
	29	+```python
	30	+def __init__(
	31	+ self,
	32	+ provider_manager: Optional[ProviderManager] = None,
	33	+ confidence_threshold: float = 0.3,
	34	+)
	35	+```
	36	+
	37	+\| Parameter \| Type \| Default \| Description \|
	38	+\|---\|---\|---\|---\|
	39	+\| `provider_manager` \| `Optional[ProviderManager]` \| `None` \| LLM provider (creates a default if not provided) \|
	40	+\| `confidence_threshold` \| `float` \| `0.3` \| Minimum confidence to process a frame at all \|
	41	+
	42	+### classify_frame()
	43	+
	44	+```python
	45	+def classify_frame(self, image_path: Union[str, Path]) -> dict
	46	+```
	47	+
	48	+Classify a single frame using a vision model. Determines whether the frame contains a diagram, slide, or other visual content worth extracting.
	49	+
	50	+Parameters:
	51	+
	52	+\| Parameter \| Type \| Description \|
	53	+\|---\|---\|---\|
	54	+\| `image_path` \| `Union[str, Path]` \| Path to the frame image file \|
	55	+
	56	+Returns: `dict` with the following keys:
	57	+
	58	+\| Key \| Type \| Description \|
	59	+\|---\|---\|---\|
	60	+\| `is_diagram` \| `bool` \| Whether the frame contains extractable content \|
	61	+\| `diagram_type` \| `str` \| One of: `flowchart`, `sequence`, `architecture`, `whiteboard`, `chart`, `table`, `slide`, `screenshot`, `unknown` \|
	62	+\| `confidence` \| `float` \| Detection confidence from 0.0 to 1.0 \|
	63	+\| `content_type` \| `str` \| Content category: `slide`, `diagram`, `document`, `screen_share`, `whiteboard`, `chart`, `person`, `other` \|
	64	+\| `brief_description` \| `str` \| One-sentence description of the frame content \|
	65	+
	66	+Important: Frames showing people, webcam feeds, or video conference participant views return `confidence: 0.0`. The classifier is tuned to detect only shared/presented content.
	67	+
	68	+```python
	69	+analyzer = DiagramAnalyzer()
	70	+result = analyzer.classify_frame("/path/to/frame_042.jpg")
	71	+if result["confidence"] >= 0.7:
	72	+ print(f"Diagram detected: {result['diagram_type']}")
	73	+```
	74	+
	75	+### analyze_diagram_single_pass()
	76	+
	77	+```python
	78	+def analyze_diagram_single_pass(self, image_path: Union[str, Path]) -> dict
	79	+```
	80	+
	81	+Full single-pass diagram analysis. Extracts description, text content, elements, relationships, Mermaid syntax, and chart data in a single LLM call.
	82	+
	83	+Returns: `dict` with the following keys:
	84	+
	85	+\| Key \| Type \| Description \|
	86	+\|---\|---\|---\|
	87	+\| `diagram_type` \| `str` \| Diagram classification \|
	88	+\| `description` \| `str` \| Detailed description of the visual content \|
	89	+\| `text_content` \| `str` \| All visible text, preserving structure \|
	90	+\| `elements` \| `list[str]` \| Identified elements/components \|
	91	+\| `relationships` \| `list[str]` \| Relationships in `"A -> B: label"` format \|
	92	+\| `mermaid` \| `str` \| Valid Mermaid diagram syntax \|
	93	+\| `chart_data` \| `dict \\| None` \| Chart data with `labels`, `values`, `chart_type` (only for data charts) \|
	94	+
	95	+Returns an empty `dict` on failure.
	96	+
	97	+### caption_frame()
	98	+
	99	+```python
	100	+def caption_frame(self, image_path: Union[str, Path]) -> str
	101	+```
	102	+
	103	+Get a brief 1-2 sentence caption for a frame. Used as a fallback when full diagram analysis is not warranted.
	104	+
	105	+Returns: `str` -- a brief description of the frame content.
	106	+
	107	+### process_frames()
	108	+
	109	+```python
	110	+def process_frames(
	111	+ self,
	112	+ frame_paths: List[Union[str, Path]],
	113	+ diagrams_dir: Optional[Path] = None,
	114	+ captures_dir: Optional[Path] = None,
	115	+) -> Tuple[List[DiagramResult], List[ScreenCapture]]
	116	+```
	117	+
	118	+Process a batch of extracted video frames through the full classification and analysis pipeline.
	119	+
	120	+Parameters:
	121	+
	122	+\| Parameter \| Type \| Default \| Description \|
	123	+\|---\|---\|---\|---\|
	124	+\| `frame_paths` \| `List[Union[str, Path]]` \| required \| Paths to frame images \|
	125	+\| `diagrams_dir` \| `Optional[Path]` \| `None` \| Output directory for diagram files (images, mermaid, JSON) \|
	126	+\| `captures_dir` \| `Optional[Path]` \| `None` \| Output directory for screengrab fallback files \|
	127	+
	128	+Returns: `Tuple[List[DiagramResult], List[ScreenCapture]]`
	129	+
	130	+Confidence thresholds:
	131	+
	132	+\| Confidence Range \| Action \|
	133	+\|---\|---\|
	134	+\| >= 0.7 \| Full diagram analysis -- extracts elements, relationships, Mermaid syntax \|
	135	+\| 0.3 to 0.7 \| Screengrab fallback -- saves frame with a brief caption \|
	136	+\| < 0.3 \| Skipped entirely \|
	137	+
	138	+Output files (when directories are provided):
	139	+
	140	+For diagrams (`diagrams_dir`):
	141	+
	142	+- `diagram_N.jpg` -- original frame image
	143	+- `diagram_N.mermaid` -- Mermaid source (if generated)
	144	+- `diagram_N.json` -- full DiagramResult as JSON
	145	+
	146	+For screen captures (`captures_dir`):
	147	+
	148	+- `capture_N.jpg` -- original frame image
	149	+- `capture_N.json` -- ScreenCapture metadata as JSON
	150	+
	151	+```python
	152	+from pathlib import Path
	153	+from video_processor.analyzers.diagram_analyzer import DiagramAnalyzer
	154	+from video_processor.providers.manager import ProviderManager
	155	+
	156	+analyzer = DiagramAnalyzer(
	157	+ provider_manager=ProviderManager(),
	158	+ confidence_threshold=0.3,
	159	+)
	160	+
	161	+frame_paths = list(Path("output/frames").glob("*.jpg"))
	162	+diagrams, captures = analyzer.process_frames(
	163	+ frame_paths,
	164	+ diagrams_dir=Path("output/diagrams"),
	165	+ captures_dir=Path("output/captures"),
	166	+)
	167	+
	168	+print(f"Found {len(diagrams)} diagrams, {len(captures)} screengrabs")
	169	+for d in diagrams:
	170	+ print(f" [{d.diagram_type.value}] {d.description}")
	171	+```
	172	+
	173	+---
	174	+
	175	+## ContentAnalyzer
	176	+
	177	+```python
	178	+from video_processor.analyzers.content_analyzer import ContentAnalyzer
	179	+```
	180	+
	181	+Cross-references transcript and diagram entities for richer knowledge extraction. Merges entities found in different sources and enriches key points with diagram links.
	182	+
	183	+### Constructor
	184	+
	185	+```python
	186	+def __init__(self, provider_manager: Optional[ProviderManager] = None)
	187	+```
	188	+
	189	+\| Parameter \| Type \| Default \| Description \|
	190	+\|---\|---\|---\|---\|
	191	+\| `provider_manager` \| `Optional[ProviderManager]` \| `None` \| Required for LLM-based fuzzy matching \|
	192	+
	193	+### cross_reference()
	194	+
	195	+```python
	196	+def cross_reference(
	197	+ self,
	198	+ transcript_entities: List[Entity],
	199	+ diagram_entities: List[Entity],
	200	+) -> List[Entity]
	201	+```
	202	+
	203	+Merge entities from transcripts and diagrams into a unified list with source attribution.
	204	+
	205	+Merge strategy:
	206	+
	207	+1. Index all transcript entities by lowercase name, marked with `source="transcript"`
	208	+2. Merge diagram entities: if a name matches, set `source="both"` and combine descriptions/occurrences; otherwise add as `source="diagram"`
	209	+3. If a `ProviderManager` is available, use LLM fuzzy matching to find additional matches among unmatched entities (e.g., "PostgreSQL" from transcript matching "Postgres" from diagram)
	210	+
	211	+Parameters:
	212	+
	213	+\| Parameter \| Type \| Description \|
	214	+\|---\|---\|---\|
	215	+\| `transcript_entities` \| `List[Entity]` \| Entities extracted from transcript \|
	216	+\| `diagram_entities` \| `List[Entity]` \| Entities extracted from diagrams \|
	217	+
	218	+Returns: `List[Entity]` -- merged entity list with `source` attribution.
	219	+
	220	+```python
	221	+from video_processor.analyzers.content_analyzer import ContentAnalyzer
	222	+from video_processor.models import Entity
	223	+
	224	+analyzer = ContentAnalyzer(provider_manager=pm)
	225	+
	226	+transcript_entities = [
	227	+ Entity(name="PostgreSQL", type="technology"),
	228	+ Entity(name="Alice", type="person"),
	229	+]
	230	+diagram_entities = [
	231	+ Entity(name="Postgres", type="technology"),
	232	+ Entity(name="Redis", type="technology"),
	233	+]
	234	+
	235	+merged = analyzer.cross_reference(transcript_entities, diagram_entities)
	236	+# "PostgreSQL" and "Postgres" may be fuzzy-matched and merged
	237	+```
	238	+
	239	+### enrich_key_points()
	240	+
	241	+```python
	242	+def enrich_key_points(
	243	+ self,
	244	+ key_points: List[KeyPoint],
	245	+ diagrams: list,
	246	+ transcript_text: str,
	247	+) -> List[KeyPoint]
	248	+```
	249	+
	250	+Link key points to relevant diagrams by entity overlap. Examines word overlap between key point text and diagram elements/text content.
	251	+
	252	+Parameters:
	253	+
	254	+\| Parameter \| Type \| Description \|
	255	+\|---\|---\|---\|
	256	+\| `key_points` \| `List[KeyPoint]` \| Key points to enrich \|
	257	+\| `diagrams` \| `list` \| List of `DiagramResult` objects or dicts \|
	258	+\| `transcript_text` \| `str` \| Full transcript text (reserved for future use) \|
	259	+
	260	+Returns: `List[KeyPoint]` -- key points with `related_diagrams` indices populated.
	261	+
	262	+A key point is linked to a diagram when they share 2 or more words (excluding short words) between the key point text/details and the diagram's elements/text content.
	263	+
	264	+---
	265	+
	266	+## ActionDetector
	267	+
	268	+```python
	269	+from video_processor.analyzers.action_detector import ActionDetector
	270	+```
	271	+
	272	+Detects action items from transcripts and diagram content using LLM extraction with a regex pattern fallback.
	273	+
	274	+### Constructor
	275	+
	276	+```python
	277	+def __init__(self, provider_manager: Optional[ProviderManager] = None)
	278	+```
	279	+
	280	+\| Parameter \| Type \| Default \| Description \|
	281	+\|---\|---\|---\|---\|
	282	+\| `provider_manager` \| `Optional[ProviderManager]` \| `None` \| Required for LLM-based extraction \|
	283	+
	284	+### detect_from_transcript()
	285	+
	286	+```python
	287	+def detect_from_transcript(
	288	+ self,
	289	+ text: str,
	290	+ segments: Optional[List[TranscriptSegment]] = None,
	291	+) -> List[ActionItem]
	292	+```
	293	+
	294	+Detect action items from transcript text.
	295	+
	296	+Parameters:
	297	+
	298	+\| Parameter \| Type \| Default \| Description \|
	299	+\|---\|---\|---\|---\|
	300	+\| `text` \| `str` \| required \| Transcript text to analyze \|
	301	+\| `segments` \| `Optional[List[TranscriptSegment]]` \| `None` \| Transcript segments for timestamp attachment \|
	302	+
	303	+Returns: `List[ActionItem]` -- detected action items with `source="transcript"`.
	304	+
	305	+Extraction modes:
	306	+
	307	+- LLM mode (when `provider_manager` is set): Sends the transcript to the LLM with a structured extraction prompt. Extracts action, assignee, deadline, priority, and context.
	308	+- Pattern mode (fallback): Matches sentences against regex patterns for action-oriented language.
	309	+
	310	+Pattern matching detects sentences containing:
	311	+
	312	+- "need/needs to", "should/must/shall"
	313	+- "will/going to", "action item/todo/follow-up"
	314	+- "assigned to/responsible for", "deadline/due by"
	315	+- "let's/let us", "make sure/ensure"
	316	+- "can you/could you/please"
	317	+
	318	+Timestamp attachment: When `segments` are provided, each action item is matched to the most relevant transcript segment (by word overlap, minimum 3 matching words), and a timestamp is added to `context`.
	319	+
	320	+### detect_from_diagrams()
	321	+
	322	+```python
	323	+def detect_from_diagrams(self, diagrams: list) -> List[ActionItem]
	324	+```
	325	+
	326	+Extract action items from diagram text content and elements. Processes each diagram's combined text using either LLM or pattern extraction.
	327	+
	328	+Parameters:
	329	+
	330	+\| Parameter \| Type \| Description \|
	331	+\|---\|---\|---\|
	332	+\| `diagrams` \| `list` \| List of `DiagramResult` objects or dicts \|
	333	+
	334	+Returns: `List[ActionItem]` -- action items with `source="diagram"`.
	335	+
	336	+### merge_action_items()
	337	+
	338	+```python
	339	+def merge_action_items(
	340	+ self,
	341	+ transcript_items: List[ActionItem],
	342	+ diagram_items: List[ActionItem],
	343	+) -> List[ActionItem]
	344	+```
	345	+
	346	+Merge action items from multiple sources, deduplicating by action text (case-insensitive, whitespace-normalized).
	347	+
	348	+Returns: `List[ActionItem]` -- deduplicated merged list.
	349	+
	350	+### Usage example
	351	+
	352	+```python
	353	+from video_processor.analyzers.action_detector import ActionDetector
	354	+from video_processor.providers.manager import ProviderManager
	355	+
	356	+detector = ActionDetector(provider_manager=ProviderManager())
	357	+
	358	+# From transcript
	359	+transcript_items = detector.detect_from_transcript(
	360	+ text="Alice needs to update the API docs by Friday. "
	361	+ "Bob should review the PR before merging.",
	362	+ segments=transcript_segments,
	363	+)
	364	+
	365	+# From diagrams
	366	+diagram_items = detector.detect_from_diagrams(diagram_results)
	367	+
	368	+# Merge and deduplicate
	369	+all_items = detector.merge_action_items(transcript_items, diagram_items)
	370	+
	371	+for item in all_items:
	372	+ print(f"[{item.priority or 'unset'}] {item.action}")
	373	+ if item.assignee:
	374	+ print(f" Assignee: {item.assignee}")
	375	+ if item.deadline:
	376	+ print(f" Deadline: {item.deadline}")
	377	+```
	378	+
	379	+### Pattern fallback (no LLM)
	380	+
	381	+```python
	382	+# Works without any API keys
	383	+detector = ActionDetector() # No provider_manager
	384	+items = detector.detect_from_transcript(
	385	+ "We need to finalize the database schema. "
	386	+ "Please update the deployment scripts."
	387	+)
	388	+# Returns ActionItems matched by regex patterns
	389	+```
8	390
9	391	ADDED docs/api/auth.md

	--- docs/api/analyzers.md
	+++ docs/api/analyzers.md
	@@ -3,5 +3,387 @@
3	::: video_processor.analyzers.diagram_analyzer
4
5	::: video_processor.analyzers.content_analyzer
6
7	::: video_processor.analyzers.action_detector






























































































































































































































































































































































































8
9	DDED docs/api/auth.md

	--- docs/api/analyzers.md
	+++ docs/api/analyzers.md
	@@ -3,5 +3,387 @@
3	::: video_processor.analyzers.diagram_analyzer
4
5	::: video_processor.analyzers.content_analyzer
6
7	::: video_processor.analyzers.action_detector
8
9	---
10
11	## Overview
12
13	The analyzers module contains the core content extraction logic for PlanOpticon. These analyzers process video frames and transcripts to extract structured knowledge: diagrams, key points, action items, and cross-referenced entities.
14
15	All analyzers accept an optional `ProviderManager` instance. When provided, they use LLM capabilities for richer extraction. Without one, they fall back to heuristic/pattern-based methods where possible.
16
17	---
18
19	## DiagramAnalyzer
20
21	```python
22	from video_processor.analyzers.diagram_analyzer import DiagramAnalyzer
23	```
24
25	Vision model-based diagram detection and analysis. Classifies video frames as diagrams, slides, screenshots, or other content, then performs full extraction on high-confidence frames.
26
27	### Constructor
28
29	```python
30	def __init__(
31	self,
32	provider_manager: Optional[ProviderManager] = None,
33	confidence_threshold: float = 0.3,
34	)
35	```
36
37	\| Parameter \| Type \| Default \| Description \|
38	\|---\|---\|---\|---\|
39	\| `provider_manager` \| `Optional[ProviderManager]` \| `None` \| LLM provider (creates a default if not provided) \|
40	\| `confidence_threshold` \| `float` \| `0.3` \| Minimum confidence to process a frame at all \|
41
42	### classify_frame()
43
44	```python
45	def classify_frame(self, image_path: Union[str, Path]) -> dict
46	```
47
48	Classify a single frame using a vision model. Determines whether the frame contains a diagram, slide, or other visual content worth extracting.
49
50	Parameters:
51
52	\| Parameter \| Type \| Description \|
53	\|---\|---\|---\|
54	\| `image_path` \| `Union[str, Path]` \| Path to the frame image file \|
55
56	Returns: `dict` with the following keys:
57
58	\| Key \| Type \| Description \|
59	\|---\|---\|---\|
60	\| `is_diagram` \| `bool` \| Whether the frame contains extractable content \|
61	\| `diagram_type` \| `str` \| One of: `flowchart`, `sequence`, `architecture`, `whiteboard`, `chart`, `table`, `slide`, `screenshot`, `unknown` \|
62	\| `confidence` \| `float` \| Detection confidence from 0.0 to 1.0 \|
63	\| `content_type` \| `str` \| Content category: `slide`, `diagram`, `document`, `screen_share`, `whiteboard`, `chart`, `person`, `other` \|
64	\| `brief_description` \| `str` \| One-sentence description of the frame content \|
65
66	Important: Frames showing people, webcam feeds, or video conference participant views return `confidence: 0.0`. The classifier is tuned to detect only shared/presented content.
67
68	```python
69	analyzer = DiagramAnalyzer()
70	result = analyzer.classify_frame("/path/to/frame_042.jpg")
71	if result["confidence"] >= 0.7:
72	print(f"Diagram detected: {result['diagram_type']}")
73	```
74
75	### analyze_diagram_single_pass()
76
77	```python
78	def analyze_diagram_single_pass(self, image_path: Union[str, Path]) -> dict
79	```
80
81	Full single-pass diagram analysis. Extracts description, text content, elements, relationships, Mermaid syntax, and chart data in a single LLM call.
82
83	Returns: `dict` with the following keys:
84
85	\| Key \| Type \| Description \|
86	\|---\|---\|---\|
87	\| `diagram_type` \| `str` \| Diagram classification \|
88	\| `description` \| `str` \| Detailed description of the visual content \|
89	\| `text_content` \| `str` \| All visible text, preserving structure \|
90	\| `elements` \| `list[str]` \| Identified elements/components \|
91	\| `relationships` \| `list[str]` \| Relationships in `"A -> B: label"` format \|
92	\| `mermaid` \| `str` \| Valid Mermaid diagram syntax \|
93	\| `chart_data` \| `dict \\| None` \| Chart data with `labels`, `values`, `chart_type` (only for data charts) \|
94
95	Returns an empty `dict` on failure.
96
97	### caption_frame()
98
99	```python
100	def caption_frame(self, image_path: Union[str, Path]) -> str
101	```
102
103	Get a brief 1-2 sentence caption for a frame. Used as a fallback when full diagram analysis is not warranted.
104
105	Returns: `str` -- a brief description of the frame content.
106
107	### process_frames()
108
109	```python
110	def process_frames(
111	self,
112	frame_paths: List[Union[str, Path]],
113	diagrams_dir: Optional[Path] = None,
114	captures_dir: Optional[Path] = None,
115	) -> Tuple[List[DiagramResult], List[ScreenCapture]]
116	```
117
118	Process a batch of extracted video frames through the full classification and analysis pipeline.
119
120	Parameters:
121
122	\| Parameter \| Type \| Default \| Description \|
123	\|---\|---\|---\|---\|
124	\| `frame_paths` \| `List[Union[str, Path]]` \| required \| Paths to frame images \|
125	\| `diagrams_dir` \| `Optional[Path]` \| `None` \| Output directory for diagram files (images, mermaid, JSON) \|
126	\| `captures_dir` \| `Optional[Path]` \| `None` \| Output directory for screengrab fallback files \|
127
128	Returns: `Tuple[List[DiagramResult], List[ScreenCapture]]`
129
130	Confidence thresholds:
131
132	\| Confidence Range \| Action \|
133	\|---\|---\|
134	\| >= 0.7 \| Full diagram analysis -- extracts elements, relationships, Mermaid syntax \|
135	\| 0.3 to 0.7 \| Screengrab fallback -- saves frame with a brief caption \|
136	\| < 0.3 \| Skipped entirely \|
137
138	Output files (when directories are provided):
139
140	For diagrams (`diagrams_dir`):
141
142	- `diagram_N.jpg` -- original frame image
143	- `diagram_N.mermaid` -- Mermaid source (if generated)
144	- `diagram_N.json` -- full DiagramResult as JSON
145
146	For screen captures (`captures_dir`):
147
148	- `capture_N.jpg` -- original frame image
149	- `capture_N.json` -- ScreenCapture metadata as JSON
150
151	```python
152	from pathlib import Path
153	from video_processor.analyzers.diagram_analyzer import DiagramAnalyzer
154	from video_processor.providers.manager import ProviderManager
155
156	analyzer = DiagramAnalyzer(
157	provider_manager=ProviderManager(),
158	confidence_threshold=0.3,
159	)
160
161	frame_paths = list(Path("output/frames").glob("*.jpg"))
162	diagrams, captures = analyzer.process_frames(
163	frame_paths,
164	diagrams_dir=Path("output/diagrams"),
165	captures_dir=Path("output/captures"),
166	)
167
168	print(f"Found {len(diagrams)} diagrams, {len(captures)} screengrabs")
169	for d in diagrams:
170	print(f" [{d.diagram_type.value}] {d.description}")
171	```
172
173	---
174
175	## ContentAnalyzer
176
177	```python
178	from video_processor.analyzers.content_analyzer import ContentAnalyzer
179	```
180
181	Cross-references transcript and diagram entities for richer knowledge extraction. Merges entities found in different sources and enriches key points with diagram links.
182
183	### Constructor
184
185	```python
186	def __init__(self, provider_manager: Optional[ProviderManager] = None)
187	```
188
189	\| Parameter \| Type \| Default \| Description \|
190	\|---\|---\|---\|---\|
191	\| `provider_manager` \| `Optional[ProviderManager]` \| `None` \| Required for LLM-based fuzzy matching \|
192
193	### cross_reference()
194
195	```python
196	def cross_reference(
197	self,
198	transcript_entities: List[Entity],
199	diagram_entities: List[Entity],
200	) -> List[Entity]
201	```
202
203	Merge entities from transcripts and diagrams into a unified list with source attribution.
204
205	Merge strategy:
206
207	1. Index all transcript entities by lowercase name, marked with `source="transcript"`
208	2. Merge diagram entities: if a name matches, set `source="both"` and combine descriptions/occurrences; otherwise add as `source="diagram"`
209	3. If a `ProviderManager` is available, use LLM fuzzy matching to find additional matches among unmatched entities (e.g., "PostgreSQL" from transcript matching "Postgres" from diagram)
210
211	Parameters:
212
213	\| Parameter \| Type \| Description \|
214	\|---\|---\|---\|
215	\| `transcript_entities` \| `List[Entity]` \| Entities extracted from transcript \|
216	\| `diagram_entities` \| `List[Entity]` \| Entities extracted from diagrams \|
217
218	Returns: `List[Entity]` -- merged entity list with `source` attribution.
219
220	```python
221	from video_processor.analyzers.content_analyzer import ContentAnalyzer
222	from video_processor.models import Entity
223
224	analyzer = ContentAnalyzer(provider_manager=pm)
225
226	transcript_entities = [
227	Entity(name="PostgreSQL", type="technology"),
228	Entity(name="Alice", type="person"),
229	]
230	diagram_entities = [
231	Entity(name="Postgres", type="technology"),
232	Entity(name="Redis", type="technology"),
233	]
234
235	merged = analyzer.cross_reference(transcript_entities, diagram_entities)
236	# "PostgreSQL" and "Postgres" may be fuzzy-matched and merged
237	```
238
239	### enrich_key_points()
240
241	```python
242	def enrich_key_points(
243	self,
244	key_points: List[KeyPoint],
245	diagrams: list,
246	transcript_text: str,
247	) -> List[KeyPoint]
248	```
249
250	Link key points to relevant diagrams by entity overlap. Examines word overlap between key point text and diagram elements/text content.
251
252	Parameters:
253
254	\| Parameter \| Type \| Description \|
255	\|---\|---\|---\|
256	\| `key_points` \| `List[KeyPoint]` \| Key points to enrich \|
257	\| `diagrams` \| `list` \| List of `DiagramResult` objects or dicts \|
258	\| `transcript_text` \| `str` \| Full transcript text (reserved for future use) \|
259
260	Returns: `List[KeyPoint]` -- key points with `related_diagrams` indices populated.
261
262	A key point is linked to a diagram when they share 2 or more words (excluding short words) between the key point text/details and the diagram's elements/text content.
263
264	---
265
266	## ActionDetector
267
268	```python
269	from video_processor.analyzers.action_detector import ActionDetector
270	```
271
272	Detects action items from transcripts and diagram content using LLM extraction with a regex pattern fallback.
273
274	### Constructor
275
276	```python
277	def __init__(self, provider_manager: Optional[ProviderManager] = None)
278	```
279
280	\| Parameter \| Type \| Default \| Description \|
281	\|---\|---\|---\|---\|
282	\| `provider_manager` \| `Optional[ProviderManager]` \| `None` \| Required for LLM-based extraction \|
283
284	### detect_from_transcript()
285
286	```python
287	def detect_from_transcript(
288	self,
289	text: str,
290	segments: Optional[List[TranscriptSegment]] = None,
291	) -> List[ActionItem]
292	```
293
294	Detect action items from transcript text.
295
296	Parameters:
297
298	\| Parameter \| Type \| Default \| Description \|
299	\|---\|---\|---\|---\|
300	\| `text` \| `str` \| required \| Transcript text to analyze \|
301	\| `segments` \| `Optional[List[TranscriptSegment]]` \| `None` \| Transcript segments for timestamp attachment \|
302
303	Returns: `List[ActionItem]` -- detected action items with `source="transcript"`.
304
305	Extraction modes:
306
307	- LLM mode (when `provider_manager` is set): Sends the transcript to the LLM with a structured extraction prompt. Extracts action, assignee, deadline, priority, and context.
308	- Pattern mode (fallback): Matches sentences against regex patterns for action-oriented language.
309
310	Pattern matching detects sentences containing:
311
312	- "need/needs to", "should/must/shall"
313	- "will/going to", "action item/todo/follow-up"
314	- "assigned to/responsible for", "deadline/due by"
315	- "let's/let us", "make sure/ensure"
316	- "can you/could you/please"
317
318	Timestamp attachment: When `segments` are provided, each action item is matched to the most relevant transcript segment (by word overlap, minimum 3 matching words), and a timestamp is added to `context`.
319
320	### detect_from_diagrams()
321
322	```python
323	def detect_from_diagrams(self, diagrams: list) -> List[ActionItem]
324	```
325
326	Extract action items from diagram text content and elements. Processes each diagram's combined text using either LLM or pattern extraction.
327
328	Parameters:
329
330	\| Parameter \| Type \| Description \|
331	\|---\|---\|---\|
332	\| `diagrams` \| `list` \| List of `DiagramResult` objects or dicts \|
333
334	Returns: `List[ActionItem]` -- action items with `source="diagram"`.
335
336	### merge_action_items()
337
338	```python
339	def merge_action_items(
340	self,
341	transcript_items: List[ActionItem],
342	diagram_items: List[ActionItem],
343	) -> List[ActionItem]
344	```
345
346	Merge action items from multiple sources, deduplicating by action text (case-insensitive, whitespace-normalized).
347
348	Returns: `List[ActionItem]` -- deduplicated merged list.
349
350	### Usage example
351
352	```python
353	from video_processor.analyzers.action_detector import ActionDetector
354	from video_processor.providers.manager import ProviderManager
355
356	detector = ActionDetector(provider_manager=ProviderManager())
357
358	# From transcript
359	transcript_items = detector.detect_from_transcript(
360	text="Alice needs to update the API docs by Friday. "
361	"Bob should review the PR before merging.",
362	segments=transcript_segments,
363	)
364
365	# From diagrams
366	diagram_items = detector.detect_from_diagrams(diagram_results)
367
368	# Merge and deduplicate
369	all_items = detector.merge_action_items(transcript_items, diagram_items)
370
371	for item in all_items:
372	print(f"[{item.priority or 'unset'}] {item.action}")
373	if item.assignee:
374	print(f" Assignee: {item.assignee}")
375	if item.deadline:
376	print(f" Deadline: {item.deadline}")
377	```
378
379	### Pattern fallback (no LLM)
380
381	```python
382	# Works without any API keys
383	detector = ActionDetector() # No provider_manager
384	items = detector.detect_from_transcript(
385	"We need to finalize the database schema. "
386	"Please update the deployment scripts."
387	)
388	# Returns ActionItems matched by regex patterns
389	```
390
391	DDED docs/api/auth.md

A docs/api/auth.md

+377

		--- a/docs/api/auth.md
		+++ b/docs/api/auth.md
		@@ -0,0 +1,377 @@
	1	+# Auth API Reference
	2	+
	3	+::: video_processor.auth
	4	+
	5	+---
	6	+
	7	+## Overview
	8	+
	9	+The `video_processor.auth` module provides a unified OAuth and authentication strategy for all PlanOpticon source connectors. It supports multiple authentication methods tried in a consistent order:
	10	+
	11	+1. Saved token -- load from disk, auto-refresh if expired
	12	+2. Client Credentials -- server-to-server OAuth (e.g., Zoom S2S)
	13	+3. OAuth 2.0 PKCE -- interactive Authorization Code flow with PKCE
	14	+4. API key fallback -- environment variable lookup
	15	+
	16	+Tokens are persisted to `~/.planopticon/` and automatically refreshed on expiry.
	17	+
	18	+---
	19	+
	20	+## AuthConfig
	21	+
	22	+```python
	23	+from video_processor.auth import AuthConfig
	24	+```
	25	+
	26	+Dataclass configuring authentication for a specific service. Defines OAuth endpoints, client credentials, API key fallback, scopes, and token storage.
	27	+
	28	+### Fields
	29	+
	30	+\| Field \| Type \| Default \| Description \|
	31	+\|---\|---\|---\|---\|
	32	+\| `service` \| `str` \| required \| Service identifier (e.g., `"zoom"`, `"notion"`) \|
	33	+\| `oauth_authorize_url` \| `Optional[str]` \| `None` \| OAuth authorization endpoint URL \|
	34	+\| `oauth_token_url` \| `Optional[str]` \| `None` \| OAuth token exchange endpoint URL \|
	35	+\| `client_id` \| `Optional[str]` \| `None` \| OAuth client ID (direct value) \|
	36	+\| `client_secret` \| `Optional[str]` \| `None` \| OAuth client secret (direct value) \|
	37	+\| `client_id_env` \| `Optional[str]` \| `None` \| Environment variable for client ID \|
	38	+\| `client_secret_env` \| `Optional[str]` \| `None` \| Environment variable for client secret \|
	39	+\| `api_key_env` \| `Optional[str]` \| `None` \| Environment variable for API key fallback \|
	40	+\| `scopes` \| `List[str]` \| `[]` \| OAuth scopes to request \|
	41	+\| `redirect_uri` \| `str` \| `"urn:ietf:wg:oauth:2.0:oob"` \| Redirect URI for auth code flow \|
	42	+\| `account_id` \| `Optional[str]` \| `None` \| Account ID for client credentials grant (direct value) \|
	43	+\| `account_id_env` \| `Optional[str]` \| `None` \| Environment variable for account ID \|
	44	+\| `token_path` \| `Optional[Path]` \| `None` \| Custom token storage path \|
	45	+
	46	+### Resolved Properties
	47	+
	48	+These properties resolve values by checking the direct field first, then falling back to the environment variable.
	49	+
	50	+\| Property \| Return Type \| Description \|
	51	+\|---\|---\|---\|
	52	+\| `resolved_client_id` \| `Optional[str]` \| Client ID from `client_id` or `os.environ[client_id_env]` \|
	53	+\| `resolved_client_secret` \| `Optional[str]` \| Client secret from `client_secret` or `os.environ[client_secret_env]` \|
	54	+\| `resolved_api_key` \| `Optional[str]` \| API key from `os.environ[api_key_env]` \|
	55	+\| `resolved_account_id` \| `Optional[str]` \| Account ID from `account_id` or `os.environ[account_id_env]` \|
	56	+\| `resolved_token_path` \| `Path` \| Token file path: `token_path` or `~/.planopticon/{service}_token.json` \|
	57	+\| `supports_oauth` \| `bool` \| `True` if both `oauth_authorize_url` and `oauth_token_url` are set \|
	58	+
	59	+```python
	60	+from video_processor.auth import AuthConfig
	61	+
	62	+config = AuthConfig(
	63	+ service="notion",
	64	+ oauth_authorize_url="https://api.notion.com/v1/oauth/authorize",
	65	+ oauth_token_url="https://api.notion.com/v1/oauth/token",
	66	+ client_id_env="NOTION_CLIENT_ID",
	67	+ client_secret_env="NOTION_CLIENT_SECRET",
	68	+ api_key_env="NOTION_API_KEY",
	69	+ scopes=["read_content"],
	70	+)
	71	+
	72	+# Check resolved values
	73	+print(config.resolved_client_id) # From NOTION_CLIENT_ID env var
	74	+print(config.supports_oauth) # True
	75	+print(config.resolved_token_path) # ~/.planopticon/notion_token.json
	76	+```
	77	+
	78	+---
	79	+
	80	+## AuthResult
	81	+
	82	+```python
	83	+from video_processor.auth import AuthResult
	84	+```
	85	+
	86	+Dataclass representing the result of an authentication attempt.
	87	+
	88	+\| Field \| Type \| Default \| Description \|
	89	+\|---\|---\|---\|---\|
	90	+\| `success` \| `bool` \| required \| Whether authentication succeeded \|
	91	+\| `access_token` \| `Optional[str]` \| `None` \| The access token (if successful) \|
	92	+\| `method` \| `Optional[str]` \| `None` \| Auth method used: `"saved_token"`, `"oauth_pkce"`, `"client_credentials"`, `"api_key"` \|
	93	+\| `expires_at` \| `Optional[float]` \| `None` \| Token expiration as Unix timestamp \|
	94	+\| `refresh_token` \| `Optional[str]` \| `None` \| OAuth refresh token (if available) \|
	95	+\| `error` \| `Optional[str]` \| `None` \| Error message (if failed) \|
	96	+
	97	+```python
	98	+result = manager.authenticate()
	99	+if result.success:
	100	+ print(f"Authenticated via {result.method}")
	101	+ print(f"Token: {result.access_token[:20]}...")
	102	+ if result.expires_at:
	103	+ import time
	104	+ remaining = result.expires_at - time.time()
	105	+ print(f"Expires in {remaining/60:.0f} minutes")
	106	+else:
	107	+ print(f"Auth failed: {result.error}")
	108	+```
	109	+
	110	+---
	111	+
	112	+## OAuthManager
	113	+
	114	+```python
	115	+from video_processor.auth import OAuthManager
	116	+```
	117	+
	118	+Manages the full authentication lifecycle for a service. Tries auth methods in priority order and handles token persistence, refresh, and PKCE flow.
	119	+
	120	+### Constructor
	121	+
	122	+```python
	123	+def __init__(self, config: AuthConfig)
	124	+```
	125	+
	126	+\| Parameter \| Type \| Description \|
	127	+\|---\|---\|---\|
	128	+\| `config` \| `AuthConfig` \| Authentication configuration for the target service \|
	129	+
	130	+### authenticate()
	131	+
	132	+```python
	133	+def authenticate(self) -> AuthResult
	134	+```
	135	+
	136	+Run the full auth chain and return the result. Methods are tried in order:
	137	+
	138	+1. Saved token -- checks `~/.planopticon/{service}_token.json`, refreshes if expired
	139	+2. Client Credentials -- if `account_id` is set and OAuth is configured, uses the client credentials grant (server-to-server)
	140	+3. OAuth PKCE -- if OAuth is configured and client ID is available, opens a browser for interactive authorization with PKCE
	141	+4. API key -- falls back to the environment variable specified in `api_key_env`
	142	+
	143	+Returns: `AuthResult` -- success/failure with token and method details.
	144	+
	145	+If all methods fail, returns an `AuthResult` with `success=False` and a helpful error message listing which environment variables to set.
	146	+
	147	+### get_token()
	148	+
	149	+```python
	150	+def get_token(self) -> Optional[str]
	151	+```
	152	+
	153	+Convenience method: run `authenticate()` and return just the access token string.
	154	+
	155	+Returns: `Optional[str]` -- the access token, or `None` if authentication failed.
	156	+
	157	+### clear_token()
	158	+
	159	+```python
	160	+def clear_token(self) -> None
	161	+```
	162	+
	163	+Remove the saved token file for this service (effectively a logout). The next `authenticate()` call will require re-authentication.
	164	+
	165	+---
	166	+
	167	+## Authentication Flows
	168	+
	169	+### Saved Token (auto-refresh)
	170	+
	171	+Tokens are saved to `~/.planopticon/{service}_token.json` as JSON. On each `authenticate()` call, the saved token is loaded and checked:
	172	+
	173	+- If the token has not expired (`time.time() < expires_at`), it is returned immediately
	174	+- If expired but a refresh token is available, the manager attempts to refresh using the OAuth token endpoint
	175	+- The refreshed token is saved back to disk
	176	+
	177	+### Client Credentials Grant
	178	+
	179	+Used for server-to-server authentication (e.g., Zoom Server-to-Server OAuth). Requires `account_id`, `client_id`, and `client_secret`. Sends a POST to the token endpoint with `grant_type=account_credentials`.
	180	+
	181	+### OAuth 2.0 Authorization Code with PKCE
	182	+
	183	+Interactive flow for user authentication:
	184	+
	185	+1. Generates a PKCE code verifier and S256 challenge
	186	+2. Constructs the authorization URL with client ID, redirect URI, scopes, and PKCE challenge
	187	+3. Opens the URL in the user's browser
	188	+4. Prompts the user to paste the authorization code
	189	+5. Exchanges the code for tokens at the token endpoint
	190	+6. Saves the tokens to disk
	191	+
	192	+### API Key Fallback
	193	+
	194	+If no OAuth flow succeeds, falls back to checking the environment variable specified in `api_key_env`. Returns the value directly as the access token.
	195	+
	196	+---
	197	+
	198	+## KNOWN_CONFIGS
	199	+
	200	+```python
	201	+from video_processor.auth import KNOWN_CONFIGS
	202	+```
	203	+
	204	+Pre-built `AuthConfig` instances for supported services. These cover the most common cloud integrations and can be used directly or as templates for custom configurations.
	205	+
	206	+\| Service Key \| Service \| OAuth Endpoints \| Client ID Env \| API Key Env \|
	207	+\|---\|---\|---\|---\|---\|
	208	+\| `"zoom"` \| Zoom \| `zoom.us/oauth/...` \| `ZOOM_CLIENT_ID` \| -- \|
	209	+\| `"notion"` \| Notion \| `api.notion.com/v1/oauth/...` \| `NOTION_CLIENT_ID` \| `NOTION_API_KEY` \|
	210	+\| `"dropbox"` \| Dropbox \| `dropbox.com/oauth2/...` \| `DROPBOX_APP_KEY` \| `DROPBOX_ACCESS_TOKEN` \|
	211	+\| `"github"` \| GitHub \| `github.com/login/oauth/...` \| `GITHUB_CLIENT_ID` \| `GITHUB_TOKEN` \|
	212	+\| `"google"` \| Google \| `accounts.google.com/o/oauth2/...` \| `GOOGLE_CLIENT_ID` \| `GOOGLE_API_KEY` \|
	213	+\| `"microsoft"` \| Microsoft \| `login.microsoftonline.com/.../oauth2/...` \| `MICROSOFT_CLIENT_ID` \| -- \|
	214	+
	215	+### Zoom
	216	+
	217	+Supports both Server-to-Server (via `ZOOM_ACCOUNT_ID`) and OAuth PKCE flows.
	218	+
	219	+```bash
	220	+# Server-to-Server
	221	+export ZOOM_CLIENT_ID="..."
	222	+export ZOOM_CLIENT_SECRET="..."
	223	+export ZOOM_ACCOUNT_ID="..."
	224	+
	225	+# Or interactive OAuth (omit ZOOM_ACCOUNT_ID)
	226	+export ZOOM_CLIENT_ID="..."
	227	+export ZOOM_CLIENT_SECRET="..."
	228	+```
	229	+
	230	+### Google (Drive, Meet, Workspace)
	231	+
	232	+Supports OAuth PKCE and API key fallback. Scopes include Drive and Docs read-only access.
	233	+
	234	+```bash
	235	+export GOOGLE_CLIENT_ID="..."
	236	+export GOOGLE_CLIENT_SECRET="..."
	237	+# Or for API-key-only access:
	238	+export GOOGLE_API_KEY="..."
	239	+```
	240	+
	241	+### GitHub
	242	+
	243	+Supports OAuth PKCE and personal access token. Requests `repo` and `read:org` scopes.
	244	+
	245	+```bash
	246	+# OAuth
	247	+export GITHUB_CLIENT_ID="..."
	248	+export GITHUB_CLIENT_SECRET="..."
	249	+# Or personal access token
	250	+export GITHUB_TOKEN="ghp_..."
	251	+```
	252	+
	253	+---
	254	+
	255	+## Helper Functions
	256	+
	257	+### get_auth_config()
	258	+
	259	+```python
	260	+def get_auth_config(service: str) -> Optional[AuthConfig]
	261	+```
	262	+
	263	+Get a pre-built `AuthConfig` for a known service.
	264	+
	265	+Parameters:
	266	+
	267	+\| Parameter \| Type \| Description \|
	268	+\|---\|---\|---\|
	269	+\| `service` \| `str` \| Service name (e.g., `"zoom"`, `"notion"`, `"github"`) \|
	270	+
	271	+Returns: `Optional[AuthConfig]` -- the config, or `None` if the service is not in `KNOWN_CONFIGS`.
	272	+
	273	+### get_auth_manager()
	274	+
	275	+```python
	276	+def get_auth_manager(service: str) -> Optional[OAuthManager]
	277	+```
	278	+
	279	+Get an `OAuthManager` for a known service. Convenience wrapper that looks up the config and creates the manager in one call.
	280	+
	281	+Returns: `Optional[OAuthManager]` -- the manager, or `None` if the service is not known.
	282	+
	283	+---
	284	+
	285	+## Usage Examples
	286	+
	287	+### Quick authentication for a known service
	288	+
	289	+```python
	290	+from video_processor.auth import get_auth_manager
	291	+
	292	+manager = get_auth_manager("zoom")
	293	+if manager:
	294	+ result = manager.authenticate()
	295	+ if result.success:
	296	+ print(f"Authenticated via {result.method}")
	297	+ # Use result.access_token for API calls
	298	+ else:
	299	+ print(f"Failed: {result.error}")
	300	+```
	301	+
	302	+### Custom service configuration
	303	+
	304	+```python
	305	+from video_processor.auth import AuthConfig, OAuthManager
	306	+
	307	+config = AuthConfig(
	308	+ service="my_service",
	309	+ oauth_authorize_url="https://my-service.com/oauth/authorize",
	310	+ oauth_token_url="https://my-service.com/oauth/token",
	311	+ client_id_env="MY_SERVICE_CLIENT_ID",
	312	+ client_secret_env="MY_SERVICE_CLIENT_SECRET",
	313	+ api_key_env="MY_SERVICE_API_KEY",
	314	+ scopes=["read", "write"],
	315	+)
	316	+
	317	+manager = OAuthManager(config)
	318	+token = manager.get_token() # Returns str or None
	319	+```
	320	+
	321	+### Using auth in a custom source connector
	322	+
	323	+```python
	324	+from pathlib import Path
	325	+from typing import List, Optional
	326	+
	327	+from video_processor.auth import OAuthManager, AuthConfig
	328	+from video_processor.sources.base import BaseSource, SourceFile
	329	+
	330	+class CustomSource(BaseSource):
	331	+ def __init__(self):
	332	+ self._config = AuthConfig(
	333	+ service="custom",
	334	+ api_key_env="CUSTOM_API_KEY",
	335	+ )
	336	+ self._manager = OAuthManager(self._config)
	337	+ self._token: Optional[str] = None
	338	+
	339	+ def authenticate(self) -> bool:
	340	+ self._token = self._manager.get_token()
	341	+ return self._token is not None
	342	+
	343	+ def list_videos(self, **kwargs) -> List[SourceFile]:
	344	+ # Use self._token to query the API
	345	+ ...
	346	+
	347	+ def download(self, file: SourceFile, destination: Path) -> Path:
	348	+ # Use self._token for authenticated downloads
	349	+ ...
	350	+```
	351	+
	352	+### Logout / clear saved token
	353	+
	354	+```python
	355	+from video_processor.auth import get_auth_manager
	356	+
	357	+manager = get_auth_manager("zoom")
	358	+if manager:
	359	+ manager.clear_token()
	360	+ print("Zoom token cleared")
	361	+```
	362	+
	363	+### Token storage location
	364	+
	365	+All tokens are stored under `~/.planopticon/`:
	366	+
	367	+```
	368	+~/.planopticon/
	369	+ zoom_token.json
	370	+ notion_token.json
	371	+ github_token.json
	372	+ google_token.json
	373	+ microsoft_token.json
	374	+ dropbox_token.json
	375	+```
	376	+
	377	+Each file contains a JSON object with `access_token`, `refresh_token` (if applicable), `expires_at`, and client credentials for refresh.

	--- a/docs/api/auth.md
	+++ b/docs/api/auth.md
	@@ -0,0 +1,377 @@

	--- a/docs/api/auth.md
	+++ b/docs/api/auth.md
	@@ -0,0 +1,377 @@
1	# Auth API Reference
2
3	::: video_processor.auth
4
5	---
6
7	## Overview
8
9	The `video_processor.auth` module provides a unified OAuth and authentication strategy for all PlanOpticon source connectors. It supports multiple authentication methods tried in a consistent order:
10
11	1. Saved token -- load from disk, auto-refresh if expired
12	2. Client Credentials -- server-to-server OAuth (e.g., Zoom S2S)
13	3. OAuth 2.0 PKCE -- interactive Authorization Code flow with PKCE
14	4. API key fallback -- environment variable lookup
15
16	Tokens are persisted to `~/.planopticon/` and automatically refreshed on expiry.
17
18	---
19
20	## AuthConfig
21
22	```python
23	from video_processor.auth import AuthConfig
24	```
25
26	Dataclass configuring authentication for a specific service. Defines OAuth endpoints, client credentials, API key fallback, scopes, and token storage.
27
28	### Fields
29
30	\| Field \| Type \| Default \| Description \|
31	\|---\|---\|---\|---\|
32	\| `service` \| `str` \| required \| Service identifier (e.g., `"zoom"`, `"notion"`) \|
33	\| `oauth_authorize_url` \| `Optional[str]` \| `None` \| OAuth authorization endpoint URL \|
34	\| `oauth_token_url` \| `Optional[str]` \| `None` \| OAuth token exchange endpoint URL \|
35	\| `client_id` \| `Optional[str]` \| `None` \| OAuth client ID (direct value) \|
36	\| `client_secret` \| `Optional[str]` \| `None` \| OAuth client secret (direct value) \|
37	\| `client_id_env` \| `Optional[str]` \| `None` \| Environment variable for client ID \|
38	\| `client_secret_env` \| `Optional[str]` \| `None` \| Environment variable for client secret \|
39	\| `api_key_env` \| `Optional[str]` \| `None` \| Environment variable for API key fallback \|
40	\| `scopes` \| `List[str]` \| `[]` \| OAuth scopes to request \|
41	\| `redirect_uri` \| `str` \| `"urn:ietf:wg:oauth:2.0:oob"` \| Redirect URI for auth code flow \|
42	\| `account_id` \| `Optional[str]` \| `None` \| Account ID for client credentials grant (direct value) \|
43	\| `account_id_env` \| `Optional[str]` \| `None` \| Environment variable for account ID \|
44	\| `token_path` \| `Optional[Path]` \| `None` \| Custom token storage path \|
45
46	### Resolved Properties
47
48	These properties resolve values by checking the direct field first, then falling back to the environment variable.
49
50	\| Property \| Return Type \| Description \|
51	\|---\|---\|---\|
52	\| `resolved_client_id` \| `Optional[str]` \| Client ID from `client_id` or `os.environ[client_id_env]` \|
53	\| `resolved_client_secret` \| `Optional[str]` \| Client secret from `client_secret` or `os.environ[client_secret_env]` \|
54	\| `resolved_api_key` \| `Optional[str]` \| API key from `os.environ[api_key_env]` \|
55	\| `resolved_account_id` \| `Optional[str]` \| Account ID from `account_id` or `os.environ[account_id_env]` \|
56	\| `resolved_token_path` \| `Path` \| Token file path: `token_path` or `~/.planopticon/{service}_token.json` \|
57	\| `supports_oauth` \| `bool` \| `True` if both `oauth_authorize_url` and `oauth_token_url` are set \|
58
59	```python
60	from video_processor.auth import AuthConfig
61
62	config = AuthConfig(
63	service="notion",
64	oauth_authorize_url="https://api.notion.com/v1/oauth/authorize",
65	oauth_token_url="https://api.notion.com/v1/oauth/token",
66	client_id_env="NOTION_CLIENT_ID",
67	client_secret_env="NOTION_CLIENT_SECRET",
68	api_key_env="NOTION_API_KEY",
69	scopes=["read_content"],
70	)
71
72	# Check resolved values
73	print(config.resolved_client_id) # From NOTION_CLIENT_ID env var
74	print(config.supports_oauth) # True
75	print(config.resolved_token_path) # ~/.planopticon/notion_token.json
76	```
77
78	---
79
80	## AuthResult
81
82	```python
83	from video_processor.auth import AuthResult
84	```
85
86	Dataclass representing the result of an authentication attempt.
87
88	\| Field \| Type \| Default \| Description \|
89	\|---\|---\|---\|---\|
90	\| `success` \| `bool` \| required \| Whether authentication succeeded \|
91	\| `access_token` \| `Optional[str]` \| `None` \| The access token (if successful) \|
92	\| `method` \| `Optional[str]` \| `None` \| Auth method used: `"saved_token"`, `"oauth_pkce"`, `"client_credentials"`, `"api_key"` \|
93	\| `expires_at` \| `Optional[float]` \| `None` \| Token expiration as Unix timestamp \|
94	\| `refresh_token` \| `Optional[str]` \| `None` \| OAuth refresh token (if available) \|
95	\| `error` \| `Optional[str]` \| `None` \| Error message (if failed) \|
96
97	```python
98	result = manager.authenticate()
99	if result.success:
100	print(f"Authenticated via {result.method}")
101	print(f"Token: {result.access_token[:20]}...")
102	if result.expires_at:
103	import time
104	remaining = result.expires_at - time.time()
105	print(f"Expires in {remaining/60:.0f} minutes")
106	else:
107	print(f"Auth failed: {result.error}")
108	```
109
110	---
111
112	## OAuthManager
113
114	```python
115	from video_processor.auth import OAuthManager
116	```
117
118	Manages the full authentication lifecycle for a service. Tries auth methods in priority order and handles token persistence, refresh, and PKCE flow.
119
120	### Constructor
121
122	```python
123	def __init__(self, config: AuthConfig)
124	```
125
126	\| Parameter \| Type \| Description \|
127	\|---\|---\|---\|
128	\| `config` \| `AuthConfig` \| Authentication configuration for the target service \|
129
130	### authenticate()
131
132	```python
133	def authenticate(self) -> AuthResult
134	```
135
136	Run the full auth chain and return the result. Methods are tried in order:
137
138	1. Saved token -- checks `~/.planopticon/{service}_token.json`, refreshes if expired
139	2. Client Credentials -- if `account_id` is set and OAuth is configured, uses the client credentials grant (server-to-server)
140	3. OAuth PKCE -- if OAuth is configured and client ID is available, opens a browser for interactive authorization with PKCE
141	4. API key -- falls back to the environment variable specified in `api_key_env`
142
143	Returns: `AuthResult` -- success/failure with token and method details.
144
145	If all methods fail, returns an `AuthResult` with `success=False` and a helpful error message listing which environment variables to set.
146
147	### get_token()
148
149	```python
150	def get_token(self) -> Optional[str]
151	```
152
153	Convenience method: run `authenticate()` and return just the access token string.
154
155	Returns: `Optional[str]` -- the access token, or `None` if authentication failed.
156
157	### clear_token()
158
159	```python
160	def clear_token(self) -> None
161	```
162
163	Remove the saved token file for this service (effectively a logout). The next `authenticate()` call will require re-authentication.
164
165	---
166
167	## Authentication Flows
168
169	### Saved Token (auto-refresh)
170
171	Tokens are saved to `~/.planopticon/{service}_token.json` as JSON. On each `authenticate()` call, the saved token is loaded and checked:
172
173	- If the token has not expired (`time.time() < expires_at`), it is returned immediately
174	- If expired but a refresh token is available, the manager attempts to refresh using the OAuth token endpoint
175	- The refreshed token is saved back to disk
176
177	### Client Credentials Grant
178
179	Used for server-to-server authentication (e.g., Zoom Server-to-Server OAuth). Requires `account_id`, `client_id`, and `client_secret`. Sends a POST to the token endpoint with `grant_type=account_credentials`.
180
181	### OAuth 2.0 Authorization Code with PKCE
182
183	Interactive flow for user authentication:
184
185	1. Generates a PKCE code verifier and S256 challenge
186	2. Constructs the authorization URL with client ID, redirect URI, scopes, and PKCE challenge
187	3. Opens the URL in the user's browser
188	4. Prompts the user to paste the authorization code
189	5. Exchanges the code for tokens at the token endpoint
190	6. Saves the tokens to disk
191
192	### API Key Fallback
193
194	If no OAuth flow succeeds, falls back to checking the environment variable specified in `api_key_env`. Returns the value directly as the access token.
195
196	---
197
198	## KNOWN_CONFIGS
199
200	```python
201	from video_processor.auth import KNOWN_CONFIGS
202	```
203
204	Pre-built `AuthConfig` instances for supported services. These cover the most common cloud integrations and can be used directly or as templates for custom configurations.
205
206	\| Service Key \| Service \| OAuth Endpoints \| Client ID Env \| API Key Env \|
207	\|---\|---\|---\|---\|---\|
208	\| `"zoom"` \| Zoom \| `zoom.us/oauth/...` \| `ZOOM_CLIENT_ID` \| -- \|
209	\| `"notion"` \| Notion \| `api.notion.com/v1/oauth/...` \| `NOTION_CLIENT_ID` \| `NOTION_API_KEY` \|
210	\| `"dropbox"` \| Dropbox \| `dropbox.com/oauth2/...` \| `DROPBOX_APP_KEY` \| `DROPBOX_ACCESS_TOKEN` \|
211	\| `"github"` \| GitHub \| `github.com/login/oauth/...` \| `GITHUB_CLIENT_ID` \| `GITHUB_TOKEN` \|
212	\| `"google"` \| Google \| `accounts.google.com/o/oauth2/...` \| `GOOGLE_CLIENT_ID` \| `GOOGLE_API_KEY` \|
213	\| `"microsoft"` \| Microsoft \| `login.microsoftonline.com/.../oauth2/...` \| `MICROSOFT_CLIENT_ID` \| -- \|
214
215	### Zoom
216
217	Supports both Server-to-Server (via `ZOOM_ACCOUNT_ID`) and OAuth PKCE flows.
218
219	```bash
220	# Server-to-Server
221	export ZOOM_CLIENT_ID="..."
222	export ZOOM_CLIENT_SECRET="..."
223	export ZOOM_ACCOUNT_ID="..."
224
225	# Or interactive OAuth (omit ZOOM_ACCOUNT_ID)
226	export ZOOM_CLIENT_ID="..."
227	export ZOOM_CLIENT_SECRET="..."
228	```
229
230	### Google (Drive, Meet, Workspace)
231
232	Supports OAuth PKCE and API key fallback. Scopes include Drive and Docs read-only access.
233
234	```bash
235	export GOOGLE_CLIENT_ID="..."
236	export GOOGLE_CLIENT_SECRET="..."
237	# Or for API-key-only access:
238	export GOOGLE_API_KEY="..."
239	```
240
241	### GitHub
242
243	Supports OAuth PKCE and personal access token. Requests `repo` and `read:org` scopes.
244
245	```bash
246	# OAuth
247	export GITHUB_CLIENT_ID="..."
248	export GITHUB_CLIENT_SECRET="..."
249	# Or personal access token
250	export GITHUB_TOKEN="ghp_..."
251	```
252
253	---
254
255	## Helper Functions
256
257	### get_auth_config()
258
259	```python
260	def get_auth_config(service: str) -> Optional[AuthConfig]
261	```
262
263	Get a pre-built `AuthConfig` for a known service.
264
265	Parameters:
266
267	\| Parameter \| Type \| Description \|
268	\|---\|---\|---\|
269	\| `service` \| `str` \| Service name (e.g., `"zoom"`, `"notion"`, `"github"`) \|
270
271	Returns: `Optional[AuthConfig]` -- the config, or `None` if the service is not in `KNOWN_CONFIGS`.
272
273	### get_auth_manager()
274
275	```python
276	def get_auth_manager(service: str) -> Optional[OAuthManager]
277	```
278
279	Get an `OAuthManager` for a known service. Convenience wrapper that looks up the config and creates the manager in one call.
280
281	Returns: `Optional[OAuthManager]` -- the manager, or `None` if the service is not known.
282
283	---
284
285	## Usage Examples
286
287	### Quick authentication for a known service
288
289	```python
290	from video_processor.auth import get_auth_manager
291
292	manager = get_auth_manager("zoom")
293	if manager:
294	result = manager.authenticate()
295	if result.success:
296	print(f"Authenticated via {result.method}")
297	# Use result.access_token for API calls
298	else:
299	print(f"Failed: {result.error}")
300	```
301
302	### Custom service configuration
303
304	```python
305	from video_processor.auth import AuthConfig, OAuthManager
306
307	config = AuthConfig(
308	service="my_service",
309	oauth_authorize_url="https://my-service.com/oauth/authorize",
310	oauth_token_url="https://my-service.com/oauth/token",
311	client_id_env="MY_SERVICE_CLIENT_ID",
312	client_secret_env="MY_SERVICE_CLIENT_SECRET",
313	api_key_env="MY_SERVICE_API_KEY",
314	scopes=["read", "write"],
315	)
316
317	manager = OAuthManager(config)
318	token = manager.get_token() # Returns str or None
319	```
320
321	### Using auth in a custom source connector
322
323	```python
324	from pathlib import Path
325	from typing import List, Optional
326
327	from video_processor.auth import OAuthManager, AuthConfig
328	from video_processor.sources.base import BaseSource, SourceFile
329
330	class CustomSource(BaseSource):
331	def __init__(self):
332	self._config = AuthConfig(
333	service="custom",
334	api_key_env="CUSTOM_API_KEY",
335	)
336	self._manager = OAuthManager(self._config)
337	self._token: Optional[str] = None
338
339	def authenticate(self) -> bool:
340	self._token = self._manager.get_token()
341	return self._token is not None
342
343	def list_videos(self, **kwargs) -> List[SourceFile]:
344	# Use self._token to query the API
345	...
346
347	def download(self, file: SourceFile, destination: Path) -> Path:
348	# Use self._token for authenticated downloads
349	...
350	```
351
352	### Logout / clear saved token
353
354	```python
355	from video_processor.auth import get_auth_manager
356
357	manager = get_auth_manager("zoom")
358	if manager:
359	manager.clear_token()
360	print("Zoom token cleared")
361	```
362
363	### Token storage location
364
365	All tokens are stored under `~/.planopticon/`:
366
367	```
368	~/.planopticon/
369	zoom_token.json
370	notion_token.json
371	github_token.json
372	google_token.json
373	microsoft_token.json
374	dropbox_token.json
375	```
376
377	Each file contains a JSON object with `access_token`, `refresh_token` (if applicable), `expires_at`, and client credentials for refresh.

M docs/api/models.md

+498

		--- docs/api/models.md
		+++ docs/api/models.md
		@@ -1,3 +1,501 @@
1	1	# Models API Reference
2	2
3	3	::: video_processor.models
	4	+
	5	+---
	6	+
	7	+## Overview
	8	+
	9	+The `video_processor.models` module defines all Pydantic data models used throughout PlanOpticon for structured output, serialization, and validation. These models represent everything from individual transcript segments to complete batch processing manifests.
	10	+
	11	+All models inherit from `pydantic.BaseModel` and support JSON serialization via `.model_dump_json()` and deserialization via `.model_validate_json()`.
	12	+
	13	+---
	14	+
	15	+## Enumerations
	16	+
	17	+### DiagramType
	18	+
	19	+Types of visual content detected in video frames.
	20	+
	21	+```python
	22	+from video_processor.models import DiagramType
	23	+```
	24	+
	25	+\| Value \| Description \|
	26	+\|---\|---\|
	27	+\| `flowchart` \| Process flow or decision tree diagrams \|
	28	+\| `sequence` \| Sequence or interaction diagrams \|
	29	+\| `architecture` \| System architecture diagrams \|
	30	+\| `whiteboard` \| Whiteboard drawings or sketches \|
	31	+\| `chart` \| Data charts (bar, line, pie, scatter) \|
	32	+\| `table` \| Tabular data \|
	33	+\| `slide` \| Presentation slides \|
	34	+\| `screenshot` \| Application screenshots or screen shares \|
	35	+\| `unknown` \| Unclassified visual content \|
	36	+
	37	+### OutputFormat
	38	+
	39	+Available output formats for processing results.
	40	+
	41	+\| Value \| Description \|
	42	+\|---\|---\|
	43	+\| `markdown` \| Markdown text \|
	44	+\| `json` \| JSON data \|
	45	+\| `html` \| HTML document \|
	46	+\| `pdf` \| PDF document \|
	47	+\| `svg` \| SVG vector graphic \|
	48	+\| `png` \| PNG raster image \|
	49	+
	50	+### PlanningEntityType
	51	+
	52	+Classification types for entities in a planning taxonomy.
	53	+
	54	+\| Value \| Description \|
	55	+\|---\|---\|
	56	+\| `goal` \| Project goals or objectives \|
	57	+\| `requirement` \| Functional or non-functional requirements \|
	58	+\| `constraint` \| Limitations or constraints \|
	59	+\| `decision` \| Decisions made during planning \|
	60	+\| `risk` \| Identified risks \|
	61	+\| `assumption` \| Planning assumptions \|
	62	+\| `dependency` \| External or internal dependencies \|
	63	+\| `milestone` \| Project milestones \|
	64	+\| `task` \| Actionable tasks \|
	65	+\| `feature` \| Product features \|
	66	+
	67	+### PlanningRelationshipType
	68	+
	69	+Relationship types within a planning taxonomy.
	70	+
	71	+\| Value \| Description \|
	72	+\|---\|---\|
	73	+\| `requires` \| Entity A requires entity B \|
	74	+\| `blocked_by` \| Entity A is blocked by entity B \|
	75	+\| `has_risk` \| Entity A has an associated risk B \|
	76	+\| `depends_on` \| Entity A depends on entity B \|
	77	+\| `addresses` \| Entity A addresses entity B \|
	78	+\| `has_tradeoff` \| Entity A involves a tradeoff with entity B \|
	79	+\| `delivers` \| Entity A delivers entity B \|
	80	+\| `implements` \| Entity A implements entity B \|
	81	+\| `parent_of` \| Entity A is the parent of entity B \|
	82	+
	83	+---
	84	+
	85	+## Protocols
	86	+
	87	+### ProgressCallback
	88	+
	89	+A runtime-checkable protocol for receiving pipeline progress updates. Implement this interface to integrate custom progress reporting (e.g., web UI, logging).
	90	+
	91	+```python
	92	+from video_processor.models import ProgressCallback
	93	+
	94	+class MyProgress:
	95	+ def on_step_start(self, step: str, index: int, total: int) -> None:
	96	+ print(f"Starting {step} ({index}/{total})")
	97	+
	98	+ def on_step_complete(self, step: str, index: int, total: int) -> None:
	99	+ print(f"Completed {step} ({index}/{total})")
	100	+
	101	+ def on_progress(self, step: str, percent: float, message: str = "") -> None:
	102	+ print(f"{step}: {percent:.0f}% {message}")
	103	+
	104	+assert isinstance(MyProgress(), ProgressCallback) # True
	105	+```
	106	+
	107	+Methods:
	108	+
	109	+\| Method \| Parameters \| Description \|
	110	+\|---\|---\|---\|
	111	+\| `on_step_start` \| `step: str`, `index: int`, `total: int` \| Called when a pipeline step begins \|
	112	+\| `on_step_complete` \| `step: str`, `index: int`, `total: int` \| Called when a pipeline step finishes \|
	113	+\| `on_progress` \| `step: str`, `percent: float`, `message: str` \| Called with incremental progress updates \|
	114	+
	115	+---
	116	+
	117	+## Transcript Models
	118	+
	119	+### TranscriptSegment
	120	+
	121	+A single segment of transcribed audio with timing and optional speaker identification.
	122	+
	123	+\| Field \| Type \| Default \| Description \|
	124	+\|---\|---\|---\|---\|
	125	+\| `start` \| `float` \| required \| Start time in seconds \|
	126	+\| `end` \| `float` \| required \| End time in seconds \|
	127	+\| `text` \| `str` \| required \| Transcribed text content \|
	128	+\| `speaker` \| `Optional[str]` \| `None` \| Speaker identifier (e.g., "Speaker 1") \|
	129	+\| `confidence` \| `Optional[float]` \| `None` \| Transcription confidence score (0.0 to 1.0) \|
	130	+
	131	+```json
	132	+{
	133	+ "start": 12.5,
	134	+ "end": 15.3,
	135	+ "text": "We should migrate to the new API by next quarter.",
	136	+ "speaker": "Alice",
	137	+ "confidence": 0.95
	138	+}
	139	+```
	140	+
	141	+---
	142	+
	143	+## Content Extraction Models
	144	+
	145	+### ActionItem
	146	+
	147	+An action item extracted from transcript or diagram content.
	148	+
	149	+\| Field \| Type \| Default \| Description \|
	150	+\|---\|---\|---\|---\|
	151	+\| `action` \| `str` \| required \| The action to be taken \|
	152	+\| `assignee` \| `Optional[str]` \| `None` \| Person responsible for the action \|
	153	+\| `deadline` \| `Optional[str]` \| `None` \| Deadline or timeframe \|
	154	+\| `priority` \| `Optional[str]` \| `None` \| Priority level (e.g., "high", "medium", "low") \|
	155	+\| `context` \| `Optional[str]` \| `None` \| Additional context or notes \|
	156	+\| `source` \| `Optional[str]` \| `None` \| Where this was found: `"transcript"`, `"diagram"`, or `"both"` \|
	157	+
	158	+```json
	159	+{
	160	+ "action": "Migrate authentication service to OAuth 2.0",
	161	+ "assignee": "Bob",
	162	+ "deadline": "Q2 2026",
	163	+ "priority": "high",
	164	+ "context": "at 245s",
	165	+ "source": "transcript"
	166	+}
	167	+```
	168	+
	169	+### KeyPoint
	170	+
	171	+A key point extracted from content, optionally linked to diagrams.
	172	+
	173	+\| Field \| Type \| Default \| Description \|
	174	+\|---\|---\|---\|---\|
	175	+\| `point` \| `str` \| required \| The key point text \|
	176	+\| `topic` \| `Optional[str]` \| `None` \| Topic or category \|
	177	+\| `details` \| `Optional[str]` \| `None` \| Supporting details \|
	178	+\| `timestamp` \| `Optional[float]` \| `None` \| Timestamp in video (seconds) \|
	179	+\| `source` \| `Optional[str]` \| `None` \| Where this was found \|
	180	+\| `related_diagrams` \| `List[int]` \| `[]` \| Indices of related diagrams in the manifest \|
	181	+
	182	+```json
	183	+{
	184	+ "point": "Team decided to use FalkorDB for graph storage",
	185	+ "topic": "Architecture",
	186	+ "details": "Embedded database avoids infrastructure overhead for CLI use",
	187	+ "timestamp": 342.0,
	188	+ "source": "transcript",
	189	+ "related_diagrams": [0, 2]
	190	+}
	191	+```
	192	+
	193	+---
	194	+
	195	+## Diagram Models
	196	+
	197	+### DiagramResult
	198	+
	199	+Result from diagram extraction and analysis. Contains structured data extracted from visual content, along with paths to output files.
	200	+
	201	+\| Field \| Type \| Default \| Description \|
	202	+\|---\|---\|---\|---\|
	203	+\| `frame_index` \| `int` \| required \| Index of the source frame \|
	204	+\| `timestamp` \| `Optional[float]` \| `None` \| Timestamp in video (seconds) \|
	205	+\| `diagram_type` \| `DiagramType` \| `unknown` \| Type of diagram detected \|
	206	+\| `confidence` \| `float` \| `0.0` \| Detection confidence (0.0 to 1.0) \|
	207	+\| `description` \| `Optional[str]` \| `None` \| Detailed description of the diagram \|
	208	+\| `text_content` \| `Optional[str]` \| `None` \| All visible text, preserving structure \|
	209	+\| `elements` \| `List[str]` \| `[]` \| Identified elements or components \|
	210	+\| `relationships` \| `List[str]` \| `[]` \| Identified relationships (e.g., `"A -> B: connects"`) \|
	211	+\| `mermaid` \| `Optional[str]` \| `None` \| Mermaid syntax representation \|
	212	+\| `chart_data` \| `Optional[Dict[str, Any]]` \| `None` \| Extractable chart data (`labels`, `values`, `chart_type`) \|
	213	+\| `image_path` \| `Optional[str]` \| `None` \| Relative path to original frame image \|
	214	+\| `svg_path` \| `Optional[str]` \| `None` \| Relative path to rendered SVG \|
	215	+\| `png_path` \| `Optional[str]` \| `None` \| Relative path to rendered PNG \|
	216	+\| `mermaid_path` \| `Optional[str]` \| `None` \| Relative path to mermaid source file \|
	217	+
	218	+```json
	219	+{
	220	+ "frame_index": 5,
	221	+ "timestamp": 120.0,
	222	+ "diagram_type": "architecture",
	223	+ "confidence": 0.92,
	224	+ "description": "Microservices architecture showing API gateway, auth service, and database layer",
	225	+ "text_content": "API Gateway\nAuth Service\nUser DB\nPostgreSQL",
	226	+ "elements": ["API Gateway", "Auth Service", "User DB", "PostgreSQL"],
	227	+ "relationships": ["API Gateway -> Auth Service: authenticates", "Auth Service -> User DB: queries"],
	228	+ "mermaid": "graph LR\n A[API Gateway] --> B[Auth Service]\n B --> C[User DB]",
	229	+ "chart_data": null,
	230	+ "image_path": "diagrams/diagram_0.jpg",
	231	+ "svg_path": null,
	232	+ "png_path": null,
	233	+ "mermaid_path": "diagrams/diagram_0.mermaid"
	234	+}
	235	+```
	236	+
	237	+### ScreenCapture
	238	+
	239	+A screengrab fallback created when diagram extraction fails or confidence is too low for full analysis.
	240	+
	241	+\| Field \| Type \| Default \| Description \|
	242	+\|---\|---\|---\|---\|
	243	+\| `frame_index` \| `int` \| required \| Index of the source frame \|
	244	+\| `timestamp` \| `Optional[float]` \| `None` \| Timestamp in video (seconds) \|
	245	+\| `caption` \| `Optional[str]` \| `None` \| Brief description of the content \|
	246	+\| `image_path` \| `Optional[str]` \| `None` \| Relative path to screenshot image \|
	247	+\| `confidence` \| `float` \| `0.0` \| Detection confidence that triggered fallback \|
	248	+
	249	+```json
	250	+{
	251	+ "frame_index": 8,
	252	+ "timestamp": 195.0,
	253	+ "caption": "Code editor showing a Python function definition",
	254	+ "image_path": "captures/capture_0.jpg",
	255	+ "confidence": 0.45
	256	+}
	257	+```
	258	+
	259	+---
	260	+
	261	+## Knowledge Graph Models
	262	+
	263	+### Entity
	264	+
	265	+An entity in the knowledge graph, representing a person, concept, technology, or other named item extracted from content.
	266	+
	267	+\| Field \| Type \| Default \| Description \|
	268	+\|---\|---\|---\|---\|
	269	+\| `name` \| `str` \| required \| Entity name \|
	270	+\| `type` \| `str` \| `"concept"` \| Entity type: `"person"`, `"concept"`, `"technology"`, `"time"`, `"diagram"` \|
	271	+\| `descriptions` \| `List[str]` \| `[]` \| Accumulated descriptions of this entity \|
	272	+\| `source` \| `Optional[str]` \| `None` \| Source attribution: `"transcript"`, `"diagram"`, or `"both"` \|
	273	+\| `occurrences` \| `List[Dict[str, Any]]` \| `[]` \| Occurrences with source, timestamp, and text context \|
	274	+
	275	+```json
	276	+{
	277	+ "name": "FalkorDB",
	278	+ "type": "technology",
	279	+ "descriptions": ["Embedded graph database", "Supports Cypher queries"],
	280	+ "source": "both",
	281	+ "occurrences": [
	282	+ {"source": "transcript", "timestamp": 120.0, "text": "We chose FalkorDB for graph storage"},
	283	+ {"source": "diagram", "text": "FalkorDB Lite"}
	284	+ ]
	285	+}
	286	+```
	287	+
	288	+### Relationship
	289	+
	290	+A directed relationship between two entities in the knowledge graph.
	291	+
	292	+\| Field \| Type \| Default \| Description \|
	293	+\|---\|---\|---\|---\|
	294	+\| `source` \| `str` \| required \| Source entity name \|
	295	+\| `target` \| `str` \| required \| Target entity name \|
	296	+\| `type` \| `str` \| `"related_to"` \| Relationship type (e.g., `"uses"`, `"manages"`, `"related_to"`) \|
	297	+\| `content_source` \| `Optional[str]` \| `None` \| Content source identifier \|
	298	+\| `timestamp` \| `Optional[float]` \| `None` \| Timestamp in seconds \|
	299	+
	300	+```json
	301	+{
	302	+ "source": "PlanOpticon",
	303	+ "target": "FalkorDB",
	304	+ "type": "uses",
	305	+ "content_source": "transcript",
	306	+ "timestamp": 125.0
	307	+}
	308	+```
	309	+
	310	+### SourceRecord
	311	+
	312	+A content source registered in the knowledge graph for provenance tracking.
	313	+
	314	+\| Field \| Type \| Default \| Description \|
	315	+\|---\|---\|---\|---\|
	316	+\| `source_id` \| `str` \| required \| Unique identifier for this source \|
	317	+\| `source_type` \| `str` \| required \| Source type: `"video"`, `"document"`, `"url"`, `"api"`, `"manual"` \|
	318	+\| `title` \| `str` \| required \| Human-readable title \|
	319	+\| `path` \| `Optional[str]` \| `None` \| Local file path \|
	320	+\| `url` \| `Optional[str]` \| `None` \| URL if applicable \|
	321	+\| `mime_type` \| `Optional[str]` \| `None` \| MIME type of the source \|
	322	+\| `ingested_at` \| `str` \| auto \| ISO format ingestion timestamp (auto-generated) \|
	323	+\| `metadata` \| `Dict[str, Any]` \| `{}` \| Additional source metadata \|
	324	+
	325	+```json
	326	+{
	327	+ "source_id": "vid_abc123",
	328	+ "source_type": "video",
	329	+ "title": "Sprint Planning Meeting - Jan 15",
	330	+ "path": "/recordings/sprint-planning.mp4",
	331	+ "url": null,
	332	+ "mime_type": "video/mp4",
	333	+ "ingested_at": "2026-01-15T10:30:00",
	334	+ "metadata": {"duration": 3600, "resolution": "1920x1080"}
	335	+}
	336	+```
	337	+
	338	+### KnowledgeGraphData
	339	+
	340	+Serializable knowledge graph data containing all nodes, relationships, and source provenance.
	341	+
	342	+\| Field \| Type \| Default \| Description \|
	343	+\|---\|---\|---\|---\|
	344	+\| `nodes` \| `List[Entity]` \| `[]` \| Graph nodes/entities \|
	345	+\| `relationships` \| `List[Relationship]` \| `[]` \| Graph relationships \|
	346	+\| `sources` \| `List[SourceRecord]` \| `[]` \| Content sources for provenance tracking \|
	347	+
	348	+---
	349	+
	350	+## Planning Models
	351	+
	352	+### PlanningEntity
	353	+
	354	+An entity classified for planning purposes, with priority and status tracking.
	355	+
	356	+\| Field \| Type \| Default \| Description \|
	357	+\|---\|---\|---\|---\|
	358	+\| `name` \| `str` \| required \| Entity name \|
	359	+\| `planning_type` \| `PlanningEntityType` \| required \| Planning classification \|
	360	+\| `description` \| `str` \| `""` \| Detailed description \|
	361	+\| `priority` \| `Optional[str]` \| `None` \| Priority: `"high"`, `"medium"`, `"low"` \|
	362	+\| `status` \| `Optional[str]` \| `None` \| Status: `"identified"`, `"confirmed"`, `"resolved"` \|
	363	+\| `source_entities` \| `List[str]` \| `[]` \| Names of source KG entities this was derived from \|
	364	+\| `metadata` \| `Dict[str, Any]` \| `{}` \| Additional metadata \|
	365	+
	366	+```json
	367	+{
	368	+ "name": "Migrate to OAuth 2.0",
	369	+ "planning_type": "task",
	370	+ "description": "Replace custom auth with OAuth 2.0 across all services",
	371	+ "priority": "high",
	372	+ "status": "identified",
	373	+ "source_entities": ["OAuth", "Authentication Service"],
	374	+ "metadata": {}
	375	+}
	376	+```
	377	+
	378	+---
	379	+
	380	+## Processing and Metadata Models
	381	+
	382	+### ProcessingStats
	383	+
	384	+Statistics about a processing run, including model usage tracking.
	385	+
	386	+\| Field \| Type \| Default \| Description \|
	387	+\|---\|---\|---\|---\|
	388	+\| `start_time` \| `Optional[str]` \| `None` \| ISO format start time \|
	389	+\| `end_time` \| `Optional[str]` \| `None` \| ISO format end time \|
	390	+\| `duration_seconds` \| `Optional[float]` \| `None` \| Total processing time \|
	391	+\| `frames_extracted` \| `int` \| `0` \| Number of frames extracted from video \|
	392	+\| `people_frames_filtered` \| `int` \| `0` \| Frames filtered out (contained people/webcam) \|
	393	+\| `diagrams_detected` \| `int` \| `0` \| Number of diagrams detected \|
	394	+\| `screen_captures` \| `int` \| `0` \| Number of screen captures saved \|
	395	+\| `transcript_duration_seconds` \| `Optional[float]` \| `None` \| Duration of transcribed audio \|
	396	+\| `models_used` \| `Dict[str, str]` \| `{}` \| Map of task to model used (e.g., `{"vision": "gpt-4o"}`) \|
	397	+
	398	+### VideoMetadata
	399	+
	400	+Metadata about the source video file.
	401	+
	402	+\| Field \| Type \| Default \| Description \|
	403	+\|---\|---\|---\|---\|
	404	+\| `title` \| `str` \| required \| Video title \|
	405	+\| `source_path` \| `Optional[str]` \| `None` \| Original video file path \|
	406	+\| `duration_seconds` \| `Optional[float]` \| `None` \| Video duration in seconds \|
	407	+\| `resolution` \| `Optional[str]` \| `None` \| Video resolution (e.g., `"1920x1080"`) \|
	408	+\| `processed_at` \| `str` \| auto \| ISO format processing timestamp \|
	409	+
	410	+---
	411	+
	412	+## Manifest Models
	413	+
	414	+### VideoManifest
	415	+
	416	+The single source of truth for a video processing run. Contains all output paths, inline structured data, and processing statistics.
	417	+
	418	+\| Field \| Type \| Default \| Description \|
	419	+\|---\|---\|---\|---\|
	420	+\| `version` \| `str` \| `"1.0"` \| Manifest schema version \|
	421	+\| `video` \| `VideoMetadata` \| required \| Source video metadata \|
	422	+\| `stats` \| `ProcessingStats` \| default \| Processing statistics \|
	423	+\| `transcript_json` \| `Optional[str]` \| `None` \| Relative path to transcript JSON \|
	424	+\| `transcript_txt` \| `Optional[str]` \| `None` \| Relative path to transcript text \|
	425	+\| `transcript_srt` \| `Optional[str]` \| `None` \| Relative path to SRT subtitles \|
	426	+\| `analysis_md` \| `Optional[str]` \| `None` \| Relative path to analysis Markdown \|
	427	+\| `analysis_html` \| `Optional[str]` \| `None` \| Relative path to analysis HTML \|
	428	+\| `analysis_pdf` \| `Optional[str]` \| `None` \| Relative path to analysis PDF \|
	429	+\| `knowledge_graph_json` \| `Optional[str]` \| `None` \| Relative path to knowledge graph JSON \|
	430	+\| `knowledge_graph_db` \| `Optional[str]` \| `None` \| Relative path to knowledge graph DB \|
	431	+\| `key_points_json` \| `Optional[str]` \| `None` \| Relative path to key points JSON \|
	432	+\| `action_items_json` \| `Optional[str]` \| `None` \| Relative path to action items JSON \|
	433	+\| `key_points` \| `List[KeyPoint]` \| `[]` \| Inline key points data \|
	434	+\| `action_items` \| `List[ActionItem]` \| `[]` \| Inline action items data \|
	435	+\| `diagrams` \| `List[DiagramResult]` \| `[]` \| Inline diagram results \|
	436	+\| `screen_captures` \| `List[ScreenCapture]` \| `[]` \| Inline screen captures \|
	437	+\| `frame_paths` \| `List[str]` \| `[]` \| Relative paths to extracted frames \|
	438	+
	439	+```python
	440	+from video_processor.models import VideoManifest, VideoMetadata
	441	+
	442	+manifest = VideoManifest(
	443	+ video=VideoMetadata(title="Sprint Planning"),
	444	+ key_points=[...],
	445	+ action_items=[...],
	446	+ diagrams=[...],
	447	+)
	448	+
	449	+# Serialize to JSON
	450	+manifest.model_dump_json(indent=2)
	451	+
	452	+# Load from file
	453	+loaded = VideoManifest.model_validate_json(Path("manifest.json").read_text())
	454	+```
	455	+
	456	+### BatchVideoEntry
	457	+
	458	+Summary of a single video within a batch processing run.
	459	+
	460	+\| Field \| Type \| Default \| Description \|
	461	+\|---\|---\|---\|---\|
	462	+\| `video_name` \| `str` \| required \| Video file name \|
	463	+\| `manifest_path` \| `str` \| required \| Relative path to the video's manifest file \|
	464	+\| `status` \| `str` \| `"pending"` \| Processing status: `"pending"`, `"completed"`, `"failed"` \|
	465	+\| `error` \| `Optional[str]` \| `None` \| Error message if processing failed \|
	466	+\| `diagrams_count` \| `int` \| `0` \| Number of diagrams detected \|
	467	+\| `action_items_count` \| `int` \| `0` \| Number of action items extracted \|
	468	+\| `key_points_count` \| `int` \| `0` \| Number of key points extracted \|
	469	+\| `duration_seconds` \| `Optional[float]` \| `None` \| Processing duration \|
	470	+
	471	+### BatchManifest
	472	+
	473	+Manifest for a batch processing run across multiple videos.
	474	+
	475	+\| Field \| Type \| Default \| Description \|
	476	+\|---\|---\|---\|---\|
	477	+\| `version` \| `str` \| `"1.0"` \| Manifest schema version \|
	478	+\| `title` \| `str` \| `"Batch Processing Results"` \| Batch title \|
	479	+\| `processed_at` \| `str` \| auto \| ISO format timestamp \|
	480	+\| `stats` \| `ProcessingStats` \| default \| Aggregated processing statistics \|
	481	+\| `videos` \| `List[BatchVideoEntry]` \| `[]` \| Per-video summaries \|
	482	+\| `total_videos` \| `int` \| `0` \| Total number of videos in batch \|
	483	+\| `completed_videos` \| `int` \| `0` \| Successfully processed videos \|
	484	+\| `failed_videos` \| `int` \| `0` \| Videos that failed processing \|
	485	+\| `total_diagrams` \| `int` \| `0` \| Total diagrams across all videos \|
	486	+\| `total_action_items` \| `int` \| `0` \| Total action items across all videos \|
	487	+\| `total_key_points` \| `int` \| `0` \| Total key points across all videos \|
	488	+\| `batch_summary_md` \| `Optional[str]` \| `None` \| Relative path to batch summary Markdown \|
	489	+\| `merged_knowledge_graph_json` \| `Optional[str]` \| `None` \| Relative path to merged KG JSON \|
	490	+\| `merged_knowledge_graph_db` \| `Optional[str]` \| `None` \| Relative path to merged KG database \|
	491	+
	492	+```python
	493	+from video_processor.models import BatchManifest
	494	+
	495	+batch = BatchManifest(
	496	+ title="Weekly Recordings",
	497	+ total_videos=5,
	498	+ completed_videos=4,
	499	+ failed_videos=1,
	500	+)
	501	+```
4	502

	--- docs/api/models.md
	+++ docs/api/models.md
	@@ -1,3 +1,501 @@
1	# Models API Reference
2
3	::: video_processor.models


















































































































































































































































































































































































































































































































4

	--- docs/api/models.md
	+++ docs/api/models.md
	@@ -1,3 +1,501 @@
1	# Models API Reference
2
3	::: video_processor.models
4
5	---
6
7	## Overview
8
9	The `video_processor.models` module defines all Pydantic data models used throughout PlanOpticon for structured output, serialization, and validation. These models represent everything from individual transcript segments to complete batch processing manifests.
10
11	All models inherit from `pydantic.BaseModel` and support JSON serialization via `.model_dump_json()` and deserialization via `.model_validate_json()`.
12
13	---
14
15	## Enumerations
16
17	### DiagramType
18
19	Types of visual content detected in video frames.
20
21	```python
22	from video_processor.models import DiagramType
23	```
24
25	\| Value \| Description \|
26	\|---\|---\|
27	\| `flowchart` \| Process flow or decision tree diagrams \|
28	\| `sequence` \| Sequence or interaction diagrams \|
29	\| `architecture` \| System architecture diagrams \|
30	\| `whiteboard` \| Whiteboard drawings or sketches \|
31	\| `chart` \| Data charts (bar, line, pie, scatter) \|
32	\| `table` \| Tabular data \|
33	\| `slide` \| Presentation slides \|
34	\| `screenshot` \| Application screenshots or screen shares \|
35	\| `unknown` \| Unclassified visual content \|
36
37	### OutputFormat
38
39	Available output formats for processing results.
40
41	\| Value \| Description \|
42	\|---\|---\|
43	\| `markdown` \| Markdown text \|
44	\| `json` \| JSON data \|
45	\| `html` \| HTML document \|
46	\| `pdf` \| PDF document \|
47	\| `svg` \| SVG vector graphic \|
48	\| `png` \| PNG raster image \|
49
50	### PlanningEntityType
51
52	Classification types for entities in a planning taxonomy.
53
54	\| Value \| Description \|
55	\|---\|---\|
56	\| `goal` \| Project goals or objectives \|
57	\| `requirement` \| Functional or non-functional requirements \|
58	\| `constraint` \| Limitations or constraints \|
59	\| `decision` \| Decisions made during planning \|
60	\| `risk` \| Identified risks \|
61	\| `assumption` \| Planning assumptions \|
62	\| `dependency` \| External or internal dependencies \|
63	\| `milestone` \| Project milestones \|
64	\| `task` \| Actionable tasks \|
65	\| `feature` \| Product features \|
66
67	### PlanningRelationshipType
68
69	Relationship types within a planning taxonomy.
70
71	\| Value \| Description \|
72	\|---\|---\|
73	\| `requires` \| Entity A requires entity B \|
74	\| `blocked_by` \| Entity A is blocked by entity B \|
75	\| `has_risk` \| Entity A has an associated risk B \|
76	\| `depends_on` \| Entity A depends on entity B \|
77	\| `addresses` \| Entity A addresses entity B \|
78	\| `has_tradeoff` \| Entity A involves a tradeoff with entity B \|
79	\| `delivers` \| Entity A delivers entity B \|
80	\| `implements` \| Entity A implements entity B \|
81	\| `parent_of` \| Entity A is the parent of entity B \|
82
83	---
84
85	## Protocols
86
87	### ProgressCallback
88
89	A runtime-checkable protocol for receiving pipeline progress updates. Implement this interface to integrate custom progress reporting (e.g., web UI, logging).
90
91	```python
92	from video_processor.models import ProgressCallback
93
94	class MyProgress:
95	def on_step_start(self, step: str, index: int, total: int) -> None:
96	print(f"Starting {step} ({index}/{total})")
97
98	def on_step_complete(self, step: str, index: int, total: int) -> None:
99	print(f"Completed {step} ({index}/{total})")
100
101	def on_progress(self, step: str, percent: float, message: str = "") -> None:
102	print(f"{step}: {percent:.0f}% {message}")
103
104	assert isinstance(MyProgress(), ProgressCallback) # True
105	```
106
107	Methods:
108
109	\| Method \| Parameters \| Description \|
110	\|---\|---\|---\|
111	\| `on_step_start` \| `step: str`, `index: int`, `total: int` \| Called when a pipeline step begins \|
112	\| `on_step_complete` \| `step: str`, `index: int`, `total: int` \| Called when a pipeline step finishes \|
113	\| `on_progress` \| `step: str`, `percent: float`, `message: str` \| Called with incremental progress updates \|
114
115	---
116
117	## Transcript Models
118
119	### TranscriptSegment
120
121	A single segment of transcribed audio with timing and optional speaker identification.
122
123	\| Field \| Type \| Default \| Description \|
124	\|---\|---\|---\|---\|
125	\| `start` \| `float` \| required \| Start time in seconds \|
126	\| `end` \| `float` \| required \| End time in seconds \|
127	\| `text` \| `str` \| required \| Transcribed text content \|
128	\| `speaker` \| `Optional[str]` \| `None` \| Speaker identifier (e.g., "Speaker 1") \|
129	\| `confidence` \| `Optional[float]` \| `None` \| Transcription confidence score (0.0 to 1.0) \|
130
131	```json
132	{
133	"start": 12.5,
134	"end": 15.3,
135	"text": "We should migrate to the new API by next quarter.",
136	"speaker": "Alice",
137	"confidence": 0.95
138	}
139	```
140
141	---
142
143	## Content Extraction Models
144
145	### ActionItem
146
147	An action item extracted from transcript or diagram content.
148
149	\| Field \| Type \| Default \| Description \|
150	\|---\|---\|---\|---\|
151	\| `action` \| `str` \| required \| The action to be taken \|
152	\| `assignee` \| `Optional[str]` \| `None` \| Person responsible for the action \|
153	\| `deadline` \| `Optional[str]` \| `None` \| Deadline or timeframe \|
154	\| `priority` \| `Optional[str]` \| `None` \| Priority level (e.g., "high", "medium", "low") \|
155	\| `context` \| `Optional[str]` \| `None` \| Additional context or notes \|
156	\| `source` \| `Optional[str]` \| `None` \| Where this was found: `"transcript"`, `"diagram"`, or `"both"` \|
157
158	```json
159	{
160	"action": "Migrate authentication service to OAuth 2.0",
161	"assignee": "Bob",
162	"deadline": "Q2 2026",
163	"priority": "high",
164	"context": "at 245s",
165	"source": "transcript"
166	}
167	```
168
169	### KeyPoint
170
171	A key point extracted from content, optionally linked to diagrams.
172
173	\| Field \| Type \| Default \| Description \|
174	\|---\|---\|---\|---\|
175	\| `point` \| `str` \| required \| The key point text \|
176	\| `topic` \| `Optional[str]` \| `None` \| Topic or category \|
177	\| `details` \| `Optional[str]` \| `None` \| Supporting details \|
178	\| `timestamp` \| `Optional[float]` \| `None` \| Timestamp in video (seconds) \|
179	\| `source` \| `Optional[str]` \| `None` \| Where this was found \|
180	\| `related_diagrams` \| `List[int]` \| `[]` \| Indices of related diagrams in the manifest \|
181
182	```json
183	{
184	"point": "Team decided to use FalkorDB for graph storage",
185	"topic": "Architecture",
186	"details": "Embedded database avoids infrastructure overhead for CLI use",
187	"timestamp": 342.0,
188	"source": "transcript",
189	"related_diagrams": [0, 2]
190	}
191	```
192
193	---
194
195	## Diagram Models
196
197	### DiagramResult
198
199	Result from diagram extraction and analysis. Contains structured data extracted from visual content, along with paths to output files.
200
201	\| Field \| Type \| Default \| Description \|
202	\|---\|---\|---\|---\|
203	\| `frame_index` \| `int` \| required \| Index of the source frame \|
204	\| `timestamp` \| `Optional[float]` \| `None` \| Timestamp in video (seconds) \|
205	\| `diagram_type` \| `DiagramType` \| `unknown` \| Type of diagram detected \|
206	\| `confidence` \| `float` \| `0.0` \| Detection confidence (0.0 to 1.0) \|
207	\| `description` \| `Optional[str]` \| `None` \| Detailed description of the diagram \|
208	\| `text_content` \| `Optional[str]` \| `None` \| All visible text, preserving structure \|
209	\| `elements` \| `List[str]` \| `[]` \| Identified elements or components \|
210	\| `relationships` \| `List[str]` \| `[]` \| Identified relationships (e.g., `"A -> B: connects"`) \|
211	\| `mermaid` \| `Optional[str]` \| `None` \| Mermaid syntax representation \|
212	\| `chart_data` \| `Optional[Dict[str, Any]]` \| `None` \| Extractable chart data (`labels`, `values`, `chart_type`) \|
213	\| `image_path` \| `Optional[str]` \| `None` \| Relative path to original frame image \|
214	\| `svg_path` \| `Optional[str]` \| `None` \| Relative path to rendered SVG \|
215	\| `png_path` \| `Optional[str]` \| `None` \| Relative path to rendered PNG \|
216	\| `mermaid_path` \| `Optional[str]` \| `None` \| Relative path to mermaid source file \|
217
218	```json
219	{
220	"frame_index": 5,
221	"timestamp": 120.0,
222	"diagram_type": "architecture",
223	"confidence": 0.92,
224	"description": "Microservices architecture showing API gateway, auth service, and database layer",
225	"text_content": "API Gateway\nAuth Service\nUser DB\nPostgreSQL",
226	"elements": ["API Gateway", "Auth Service", "User DB", "PostgreSQL"],
227	"relationships": ["API Gateway -> Auth Service: authenticates", "Auth Service -> User DB: queries"],
228	"mermaid": "graph LR\n A[API Gateway] --> B[Auth Service]\n B --> C[User DB]",
229	"chart_data": null,
230	"image_path": "diagrams/diagram_0.jpg",
231	"svg_path": null,
232	"png_path": null,
233	"mermaid_path": "diagrams/diagram_0.mermaid"
234	}
235	```
236
237	### ScreenCapture
238
239	A screengrab fallback created when diagram extraction fails or confidence is too low for full analysis.
240
241	\| Field \| Type \| Default \| Description \|
242	\|---\|---\|---\|---\|
243	\| `frame_index` \| `int` \| required \| Index of the source frame \|
244	\| `timestamp` \| `Optional[float]` \| `None` \| Timestamp in video (seconds) \|
245	\| `caption` \| `Optional[str]` \| `None` \| Brief description of the content \|
246	\| `image_path` \| `Optional[str]` \| `None` \| Relative path to screenshot image \|
247	\| `confidence` \| `float` \| `0.0` \| Detection confidence that triggered fallback \|
248
249	```json
250	{
251	"frame_index": 8,
252	"timestamp": 195.0,
253	"caption": "Code editor showing a Python function definition",
254	"image_path": "captures/capture_0.jpg",
255	"confidence": 0.45
256	}
257	```
258
259	---
260
261	## Knowledge Graph Models
262
263	### Entity
264
265	An entity in the knowledge graph, representing a person, concept, technology, or other named item extracted from content.
266
267	\| Field \| Type \| Default \| Description \|
268	\|---\|---\|---\|---\|
269	\| `name` \| `str` \| required \| Entity name \|
270	\| `type` \| `str` \| `"concept"` \| Entity type: `"person"`, `"concept"`, `"technology"`, `"time"`, `"diagram"` \|
271	\| `descriptions` \| `List[str]` \| `[]` \| Accumulated descriptions of this entity \|
272	\| `source` \| `Optional[str]` \| `None` \| Source attribution: `"transcript"`, `"diagram"`, or `"both"` \|
273	\| `occurrences` \| `List[Dict[str, Any]]` \| `[]` \| Occurrences with source, timestamp, and text context \|
274
275	```json
276	{
277	"name": "FalkorDB",
278	"type": "technology",
279	"descriptions": ["Embedded graph database", "Supports Cypher queries"],
280	"source": "both",
281	"occurrences": [
282	{"source": "transcript", "timestamp": 120.0, "text": "We chose FalkorDB for graph storage"},
283	{"source": "diagram", "text": "FalkorDB Lite"}
284	]
285	}
286	```
287
288	### Relationship
289
290	A directed relationship between two entities in the knowledge graph.
291
292	\| Field \| Type \| Default \| Description \|
293	\|---\|---\|---\|---\|
294	\| `source` \| `str` \| required \| Source entity name \|
295	\| `target` \| `str` \| required \| Target entity name \|
296	\| `type` \| `str` \| `"related_to"` \| Relationship type (e.g., `"uses"`, `"manages"`, `"related_to"`) \|
297	\| `content_source` \| `Optional[str]` \| `None` \| Content source identifier \|
298	\| `timestamp` \| `Optional[float]` \| `None` \| Timestamp in seconds \|
299
300	```json
301	{
302	"source": "PlanOpticon",
303	"target": "FalkorDB",
304	"type": "uses",
305	"content_source": "transcript",
306	"timestamp": 125.0
307	}
308	```
309
310	### SourceRecord
311
312	A content source registered in the knowledge graph for provenance tracking.
313
314	\| Field \| Type \| Default \| Description \|
315	\|---\|---\|---\|---\|
316	\| `source_id` \| `str` \| required \| Unique identifier for this source \|
317	\| `source_type` \| `str` \| required \| Source type: `"video"`, `"document"`, `"url"`, `"api"`, `"manual"` \|
318	\| `title` \| `str` \| required \| Human-readable title \|
319	\| `path` \| `Optional[str]` \| `None` \| Local file path \|
320	\| `url` \| `Optional[str]` \| `None` \| URL if applicable \|
321	\| `mime_type` \| `Optional[str]` \| `None` \| MIME type of the source \|
322	\| `ingested_at` \| `str` \| auto \| ISO format ingestion timestamp (auto-generated) \|
323	\| `metadata` \| `Dict[str, Any]` \| `{}` \| Additional source metadata \|
324
325	```json
326	{
327	"source_id": "vid_abc123",
328	"source_type": "video",
329	"title": "Sprint Planning Meeting - Jan 15",
330	"path": "/recordings/sprint-planning.mp4",
331	"url": null,
332	"mime_type": "video/mp4",
333	"ingested_at": "2026-01-15T10:30:00",
334	"metadata": {"duration": 3600, "resolution": "1920x1080"}
335	}
336	```
337
338	### KnowledgeGraphData
339
340	Serializable knowledge graph data containing all nodes, relationships, and source provenance.
341
342	\| Field \| Type \| Default \| Description \|
343	\|---\|---\|---\|---\|
344	\| `nodes` \| `List[Entity]` \| `[]` \| Graph nodes/entities \|
345	\| `relationships` \| `List[Relationship]` \| `[]` \| Graph relationships \|
346	\| `sources` \| `List[SourceRecord]` \| `[]` \| Content sources for provenance tracking \|
347
348	---
349
350	## Planning Models
351
352	### PlanningEntity
353
354	An entity classified for planning purposes, with priority and status tracking.
355
356	\| Field \| Type \| Default \| Description \|
357	\|---\|---\|---\|---\|
358	\| `name` \| `str` \| required \| Entity name \|
359	\| `planning_type` \| `PlanningEntityType` \| required \| Planning classification \|
360	\| `description` \| `str` \| `""` \| Detailed description \|
361	\| `priority` \| `Optional[str]` \| `None` \| Priority: `"high"`, `"medium"`, `"low"` \|
362	\| `status` \| `Optional[str]` \| `None` \| Status: `"identified"`, `"confirmed"`, `"resolved"` \|
363	\| `source_entities` \| `List[str]` \| `[]` \| Names of source KG entities this was derived from \|
364	\| `metadata` \| `Dict[str, Any]` \| `{}` \| Additional metadata \|
365
366	```json
367	{
368	"name": "Migrate to OAuth 2.0",
369	"planning_type": "task",
370	"description": "Replace custom auth with OAuth 2.0 across all services",
371	"priority": "high",
372	"status": "identified",
373	"source_entities": ["OAuth", "Authentication Service"],
374	"metadata": {}
375	}
376	```
377
378	---
379
380	## Processing and Metadata Models
381
382	### ProcessingStats
383
384	Statistics about a processing run, including model usage tracking.
385
386	\| Field \| Type \| Default \| Description \|
387	\|---\|---\|---\|---\|
388	\| `start_time` \| `Optional[str]` \| `None` \| ISO format start time \|
389	\| `end_time` \| `Optional[str]` \| `None` \| ISO format end time \|
390	\| `duration_seconds` \| `Optional[float]` \| `None` \| Total processing time \|
391	\| `frames_extracted` \| `int` \| `0` \| Number of frames extracted from video \|
392	\| `people_frames_filtered` \| `int` \| `0` \| Frames filtered out (contained people/webcam) \|
393	\| `diagrams_detected` \| `int` \| `0` \| Number of diagrams detected \|
394	\| `screen_captures` \| `int` \| `0` \| Number of screen captures saved \|
395	\| `transcript_duration_seconds` \| `Optional[float]` \| `None` \| Duration of transcribed audio \|
396	\| `models_used` \| `Dict[str, str]` \| `{}` \| Map of task to model used (e.g., `{"vision": "gpt-4o"}`) \|
397
398	### VideoMetadata
399
400	Metadata about the source video file.
401
402	\| Field \| Type \| Default \| Description \|
403	\|---\|---\|---\|---\|
404	\| `title` \| `str` \| required \| Video title \|
405	\| `source_path` \| `Optional[str]` \| `None` \| Original video file path \|
406	\| `duration_seconds` \| `Optional[float]` \| `None` \| Video duration in seconds \|
407	\| `resolution` \| `Optional[str]` \| `None` \| Video resolution (e.g., `"1920x1080"`) \|
408	\| `processed_at` \| `str` \| auto \| ISO format processing timestamp \|
409
410	---
411
412	## Manifest Models
413
414	### VideoManifest
415
416	The single source of truth for a video processing run. Contains all output paths, inline structured data, and processing statistics.
417
418	\| Field \| Type \| Default \| Description \|
419	\|---\|---\|---\|---\|
420	\| `version` \| `str` \| `"1.0"` \| Manifest schema version \|
421	\| `video` \| `VideoMetadata` \| required \| Source video metadata \|
422	\| `stats` \| `ProcessingStats` \| default \| Processing statistics \|
423	\| `transcript_json` \| `Optional[str]` \| `None` \| Relative path to transcript JSON \|
424	\| `transcript_txt` \| `Optional[str]` \| `None` \| Relative path to transcript text \|
425	\| `transcript_srt` \| `Optional[str]` \| `None` \| Relative path to SRT subtitles \|
426	\| `analysis_md` \| `Optional[str]` \| `None` \| Relative path to analysis Markdown \|
427	\| `analysis_html` \| `Optional[str]` \| `None` \| Relative path to analysis HTML \|
428	\| `analysis_pdf` \| `Optional[str]` \| `None` \| Relative path to analysis PDF \|
429	\| `knowledge_graph_json` \| `Optional[str]` \| `None` \| Relative path to knowledge graph JSON \|
430	\| `knowledge_graph_db` \| `Optional[str]` \| `None` \| Relative path to knowledge graph DB \|
431	\| `key_points_json` \| `Optional[str]` \| `None` \| Relative path to key points JSON \|
432	\| `action_items_json` \| `Optional[str]` \| `None` \| Relative path to action items JSON \|
433	\| `key_points` \| `List[KeyPoint]` \| `[]` \| Inline key points data \|
434	\| `action_items` \| `List[ActionItem]` \| `[]` \| Inline action items data \|
435	\| `diagrams` \| `List[DiagramResult]` \| `[]` \| Inline diagram results \|
436	\| `screen_captures` \| `List[ScreenCapture]` \| `[]` \| Inline screen captures \|
437	\| `frame_paths` \| `List[str]` \| `[]` \| Relative paths to extracted frames \|
438
439	```python
440	from video_processor.models import VideoManifest, VideoMetadata
441
442	manifest = VideoManifest(
443	video=VideoMetadata(title="Sprint Planning"),
444	key_points=[...],
445	action_items=[...],
446	diagrams=[...],
447	)
448
449	# Serialize to JSON
450	manifest.model_dump_json(indent=2)
451
452	# Load from file
453	loaded = VideoManifest.model_validate_json(Path("manifest.json").read_text())
454	```
455
456	### BatchVideoEntry
457
458	Summary of a single video within a batch processing run.
459
460	\| Field \| Type \| Default \| Description \|
461	\|---\|---\|---\|---\|
462	\| `video_name` \| `str` \| required \| Video file name \|
463	\| `manifest_path` \| `str` \| required \| Relative path to the video's manifest file \|
464	\| `status` \| `str` \| `"pending"` \| Processing status: `"pending"`, `"completed"`, `"failed"` \|
465	\| `error` \| `Optional[str]` \| `None` \| Error message if processing failed \|
466	\| `diagrams_count` \| `int` \| `0` \| Number of diagrams detected \|
467	\| `action_items_count` \| `int` \| `0` \| Number of action items extracted \|
468	\| `key_points_count` \| `int` \| `0` \| Number of key points extracted \|
469	\| `duration_seconds` \| `Optional[float]` \| `None` \| Processing duration \|
470
471	### BatchManifest
472
473	Manifest for a batch processing run across multiple videos.
474
475	\| Field \| Type \| Default \| Description \|
476	\|---\|---\|---\|---\|
477	\| `version` \| `str` \| `"1.0"` \| Manifest schema version \|
478	\| `title` \| `str` \| `"Batch Processing Results"` \| Batch title \|
479	\| `processed_at` \| `str` \| auto \| ISO format timestamp \|
480	\| `stats` \| `ProcessingStats` \| default \| Aggregated processing statistics \|
481	\| `videos` \| `List[BatchVideoEntry]` \| `[]` \| Per-video summaries \|
482	\| `total_videos` \| `int` \| `0` \| Total number of videos in batch \|
483	\| `completed_videos` \| `int` \| `0` \| Successfully processed videos \|
484	\| `failed_videos` \| `int` \| `0` \| Videos that failed processing \|
485	\| `total_diagrams` \| `int` \| `0` \| Total diagrams across all videos \|
486	\| `total_action_items` \| `int` \| `0` \| Total action items across all videos \|
487	\| `total_key_points` \| `int` \| `0` \| Total key points across all videos \|
488	\| `batch_summary_md` \| `Optional[str]` \| `None` \| Relative path to batch summary Markdown \|
489	\| `merged_knowledge_graph_json` \| `Optional[str]` \| `None` \| Relative path to merged KG JSON \|
490	\| `merged_knowledge_graph_db` \| `Optional[str]` \| `None` \| Relative path to merged KG database \|
491
492	```python
493	from video_processor.models import BatchManifest
494
495	batch = BatchManifest(
496	title="Weekly Recordings",
497	total_videos=5,
498	completed_videos=4,
499	failed_videos=1,
500	)
501	```
502

M docs/api/providers.md

+496

		--- docs/api/providers.md
		+++ docs/api/providers.md
		@@ -3,5 +3,501 @@
3	3	::: video_processor.providers.base
4	4
5	5	::: video_processor.providers.manager
6	6
7	7	::: video_processor.providers.discovery
	8	+
	9	+---
	10	+
	11	+## Overview
	12	+
	13	+The provider system abstracts LLM API calls behind a unified interface. It supports multiple providers (OpenAI, Anthropic, Gemini, Ollama, and OpenAI-compatible services), automatic model discovery, capability-based routing, and usage tracking.
	14	+
	15	+Key components:
	16	+
	17	+- `BaseProvider` -- abstract interface that all providers implement
	18	+- `ProviderRegistry` -- global registry mapping provider names to classes
	19	+- `ProviderManager` -- high-level router that picks the best provider for each task
	20	+- `discover_available_models()` -- scans all configured providers for available models
	21	+
	22	+---
	23	+
	24	+## BaseProvider (ABC)
	25	+
	26	+```python
	27	+from video_processor.providers.base import BaseProvider
	28	+```
	29	+
	30	+Abstract base class that all provider implementations must subclass. Defines the four core capabilities: chat, vision, audio transcription, and model listing.
	31	+
	32	+Class attribute:
	33	+
	34	+\| Attribute \| Type \| Description \|
	35	+\|---\|---\|---\|
	36	+\| `provider_name` \| `str` \| Identifier for this provider (e.g., `"openai"`, `"anthropic"`) \|
	37	+
	38	+### chat()
	39	+
	40	+```python
	41	+def chat(
	42	+ self,
	43	+ messages: list[dict],
	44	+ max_tokens: int = 4096,
	45	+ temperature: float = 0.7,
	46	+ model: Optional[str] = None,
	47	+) -> str
	48	+```
	49	+
	50	+Send a chat completion request.
	51	+
	52	+Parameters:
	53	+
	54	+\| Parameter \| Type \| Default \| Description \|
	55	+\|---\|---\|---\|---\|
	56	+\| `messages` \| `list[dict]` \| required \| OpenAI-format message list (`role`, `content`) \|
	57	+\| `max_tokens` \| `int` \| `4096` \| Maximum tokens in the response \|
	58	+\| `temperature` \| `float` \| `0.7` \| Sampling temperature \|
	59	+\| `model` \| `Optional[str]` \| `None` \| Override model ID \|
	60	+
	61	+Returns: `str` -- the assistant's text response.
	62	+
	63	+### analyze_image()
	64	+
	65	+```python
	66	+def analyze_image(
	67	+ self,
	68	+ image_bytes: bytes,
	69	+ prompt: str,
	70	+ max_tokens: int = 4096,
	71	+ model: Optional[str] = None,
	72	+) -> str
	73	+```
	74	+
	75	+Analyze an image with a text prompt using a vision-capable model.
	76	+
	77	+Parameters:
	78	+
	79	+\| Parameter \| Type \| Default \| Description \|
	80	+\|---\|---\|---\|---\|
	81	+\| `image_bytes` \| `bytes` \| required \| Raw image data (JPEG, PNG, etc.) \|
	82	+\| `prompt` \| `str` \| required \| Analysis instructions \|
	83	+\| `max_tokens` \| `int` \| `4096` \| Maximum tokens in the response \|
	84	+\| `model` \| `Optional[str]` \| `None` \| Override model ID \|
	85	+
	86	+Returns: `str` -- the assistant's analysis text.
	87	+
	88	+### transcribe_audio()
	89	+
	90	+```python
	91	+def transcribe_audio(
	92	+ self,
	93	+ audio_path: str \| Path,
	94	+ language: Optional[str] = None,
	95	+ model: Optional[str] = None,
	96	+) -> dict
	97	+```
	98	+
	99	+Transcribe an audio file.
	100	+
	101	+Parameters:
	102	+
	103	+\| Parameter \| Type \| Default \| Description \|
	104	+\|---\|---\|---\|---\|
	105	+\| `audio_path` \| `str \\| Path` \| required \| Path to the audio file \|
	106	+\| `language` \| `Optional[str]` \| `None` \| Language hint (ISO 639-1 code) \|
	107	+\| `model` \| `Optional[str]` \| `None` \| Override model ID \|
	108	+
	109	+Returns: `dict` -- transcription result with keys `text`, `segments`, `duration`, etc.
	110	+
	111	+### list_models()
	112	+
	113	+```python
	114	+def list_models(self) -> list[ModelInfo]
	115	+```
	116	+
	117	+Discover available models from this provider's API.
	118	+
	119	+Returns: `list[ModelInfo]` -- available models with capability metadata.
	120	+
	121	+---
	122	+
	123	+## ModelInfo
	124	+
	125	+```python
	126	+from video_processor.providers.base import ModelInfo
	127	+```
	128	+
	129	+Pydantic model describing an available model from a provider.
	130	+
	131	+\| Field \| Type \| Default \| Description \|
	132	+\|---\|---\|---\|---\|
	133	+\| `id` \| `str` \| required \| Model identifier (e.g., `"gpt-4o"`, `"claude-haiku-4-5-20251001"`) \|
	134	+\| `provider` \| `str` \| required \| Provider name (e.g., `"openai"`, `"anthropic"`, `"gemini"`) \|
	135	+\| `display_name` \| `str` \| `""` \| Human-readable display name \|
	136	+\| `capabilities` \| `List[str]` \| `[]` \| Model capabilities: `"chat"`, `"vision"`, `"audio"`, `"embedding"` \|
	137	+
	138	+```json
	139	+{
	140	+ "id": "gpt-4o",
	141	+ "provider": "openai",
	142	+ "display_name": "GPT-4o",
	143	+ "capabilities": ["chat", "vision"]
	144	+}
	145	+```
	146	+
	147	+---
	148	+
	149	+## ProviderRegistry
	150	+
	151	+```python
	152	+from video_processor.providers.base import ProviderRegistry
	153	+```
	154	+
	155	+Class-level registry for provider classes. Providers register themselves with metadata on import. This registry is used internally by `ProviderManager` but can also be used directly for introspection.
	156	+
	157	+### register()
	158	+
	159	+```python
	160	+@classmethod
	161	+def register(
	162	+ cls,
	163	+ name: str,
	164	+ provider_class: type,
	165	+ env_var: str = "",
	166	+ model_prefixes: Optional[List[str]] = None,
	167	+ default_models: Optional[Dict[str, str]] = None,
	168	+) -> None
	169	+```
	170	+
	171	+Register a provider class with its metadata. Called by each provider module at import time.
	172	+
	173	+Parameters:
	174	+
	175	+\| Parameter \| Type \| Default \| Description \|
	176	+\|---\|---\|---\|---\|
	177	+\| `name` \| `str` \| required \| Provider name (e.g., `"openai"`) \|
	178	+\| `provider_class` \| `type` \| required \| The provider class \|
	179	+\| `env_var` \| `str` \| `""` \| Environment variable for API key \|
	180	+\| `model_prefixes` \| `Optional[List[str]]` \| `None` \| Model ID prefixes for auto-detection (e.g., `["gpt-", "o1-"]`) \|
	181	+\| `default_models` \| `Optional[Dict[str, str]]` \| `None` \| Default models per capability (e.g., `{"chat": "gpt-4o", "vision": "gpt-4o"}`) \|
	182	+
	183	+### get()
	184	+
	185	+```python
	186	+@classmethod
	187	+def get(cls, name: str) -> type
	188	+```
	189	+
	190	+Return the provider class for a given name. Raises `ValueError` if the provider is not registered.
	191	+
	192	+### get_by_model()
	193	+
	194	+```python
	195	+@classmethod
	196	+def get_by_model(cls, model_id: str) -> Optional[str]
	197	+```
	198	+
	199	+Return the provider name for a model ID based on prefix matching. Returns `None` if no match is found.
	200	+
	201	+### get_default_models()
	202	+
	203	+```python
	204	+@classmethod
	205	+def get_default_models(cls, name: str) -> Dict[str, str]
	206	+```
	207	+
	208	+Return the default models dict for a provider, mapping capability names to model IDs.
	209	+
	210	+### available()
	211	+
	212	+```python
	213	+@classmethod
	214	+def available(cls) -> List[str]
	215	+```
	216	+
	217	+Return names of providers whose required environment variable is set (or providers with no env var requirement, like Ollama).
	218	+
	219	+### all_registered()
	220	+
	221	+```python
	222	+@classmethod
	223	+def all_registered(cls) -> Dict[str, Dict]
	224	+```
	225	+
	226	+Return all registered providers and their metadata dictionaries.
	227	+
	228	+---
	229	+
	230	+## OpenAICompatibleProvider
	231	+
	232	+```python
	233	+from video_processor.providers.base import OpenAICompatibleProvider
	234	+```
	235	+
	236	+Base class for providers using OpenAI-compatible APIs (Together, Fireworks, Cerebras, xAI, Azure). Implements `chat()`, `analyze_image()`, and `list_models()` using the OpenAI client library. `transcribe_audio()` raises `NotImplementedError` by default.
	237	+
	238	+Constructor:
	239	+
	240	+```python
	241	+def __init__(self, api_key: Optional[str] = None, base_url: Optional[str] = None)
	242	+```
	243	+
	244	+\| Parameter \| Type \| Default \| Description \|
	245	+\|---\|---\|---\|---\|
	246	+\| `api_key` \| `Optional[str]` \| `None` \| API key (falls back to `self.env_var` environment variable) \|
	247	+\| `base_url` \| `Optional[str]` \| `None` \| API base URL (falls back to `self.base_url` class attribute) \|
	248	+
	249	+Subclass attributes to override:
	250	+
	251	+\| Attribute \| Description \|
	252	+\|---\|---\|
	253	+\| `provider_name` \| Provider identifier string \|
	254	+\| `base_url` \| Default API base URL \|
	255	+\| `env_var` \| Environment variable name for the API key \|
	256	+
	257	+Usage tracking: After each `chat()` or `analyze_image()` call, the provider stores token counts in `self._last_usage` as `{"input_tokens": int, "output_tokens": int}`. This is consumed by `ProviderManager._track()`.
	258	+
	259	+---
	260	+
	261	+## ProviderManager
	262	+
	263	+```python
	264	+from video_processor.providers.manager import ProviderManager
	265	+```
	266	+
	267	+High-level router that selects the best available provider and model for each API call. Supports explicit model selection, forced provider, or automatic selection based on discovered capabilities.
	268	+
	269	+### Constructor
	270	+
	271	+```python
	272	+def __init__(
	273	+ self,
	274	+ vision_model: Optional[str] = None,
	275	+ chat_model: Optional[str] = None,
	276	+ transcription_model: Optional[str] = None,
	277	+ provider: Optional[str] = None,
	278	+ auto: bool = True,
	279	+)
	280	+```
	281	+
	282	+\| Parameter \| Type \| Default \| Description \|
	283	+\|---\|---\|---\|---\|
	284	+\| `vision_model` \| `Optional[str]` \| `None` \| Override model for vision tasks (e.g., `"gpt-4o"`) \|
	285	+\| `chat_model` \| `Optional[str]` \| `None` \| Override model for chat/LLM tasks \|
	286	+\| `transcription_model` \| `Optional[str]` \| `None` \| Override model for transcription \|
	287	+\| `provider` \| `Optional[str]` \| `None` \| Force all tasks to a single provider \|
	288	+\| `auto` \| `bool` \| `True` \| If `True` and no model specified, pick the best available \|
	289	+
	290	+Attributes:
	291	+
	292	+\| Attribute \| Type \| Description \|
	293	+\|---\|---\|---\|
	294	+\| `usage` \| `UsageTracker` \| Tracks token counts and API costs across all calls \|
	295	+
	296	+### Auto-selection preferences
	297	+
	298	+When `auto=True` and no explicit model is set, providers are tried in this order:
	299	+
	300	+Vision: Gemini (`gemini-2.5-flash`) > OpenAI (`gpt-4o-mini`) > Anthropic (`claude-haiku-4-5-20251001`)
	301	+
	302	+Chat: Anthropic (`claude-haiku-4-5-20251001`) > OpenAI (`gpt-4o-mini`) > Gemini (`gemini-2.5-flash`)
	303	+
	304	+Transcription: OpenAI (`whisper-1`) > Gemini (`gemini-2.5-flash`)
	305	+
	306	+If no API-key-based provider is available, Ollama is tried as a fallback.
	307	+
	308	+### chat()
	309	+
	310	+```python
	311	+def chat(
	312	+ self,
	313	+ messages: list[dict],
	314	+ max_tokens: int = 4096,
	315	+ temperature: float = 0.7,
	316	+) -> str
	317	+```
	318	+
	319	+Send a chat completion to the best available provider. Automatically resolves which provider and model to use.
	320	+
	321	+Parameters:
	322	+
	323	+\| Parameter \| Type \| Default \| Description \|
	324	+\|---\|---\|---\|---\|
	325	+\| `messages` \| `list[dict]` \| required \| OpenAI-format messages \|
	326	+\| `max_tokens` \| `int` \| `4096` \| Maximum response tokens \|
	327	+\| `temperature` \| `float` \| `0.7` \| Sampling temperature \|
	328	+
	329	+Returns: `str` -- assistant response text.
	330	+
	331	+Raises: `RuntimeError` if no provider is available for the `chat` capability.
	332	+
	333	+### analyze_image()
	334	+
	335	+```python
	336	+def analyze_image(
	337	+ self,
	338	+ image_bytes: bytes,
	339	+ prompt: str,
	340	+ max_tokens: int = 4096,
	341	+) -> str
	342	+```
	343	+
	344	+Analyze an image using the best available vision provider.
	345	+
	346	+Returns: `str` -- analysis text.
	347	+
	348	+Raises: `RuntimeError` if no provider is available for the `vision` capability.
	349	+
	350	+### transcribe_audio()
	351	+
	352	+```python
	353	+def transcribe_audio(
	354	+ self,
	355	+ audio_path: str \| Path,
	356	+ language: Optional[str] = None,
	357	+ speaker_hints: Optional[list[str]] = None,
	358	+) -> dict
	359	+```
	360	+
	361	+Transcribe audio. Prefers local Whisper (no file size limits, no API costs) when available, falling back to API-based transcription.
	362	+
	363	+Parameters:
	364	+
	365	+\| Parameter \| Type \| Default \| Description \|
	366	+\|---\|---\|---\|---\|
	367	+\| `audio_path` \| `str \\| Path` \| required \| Path to the audio file \|
	368	+\| `language` \| `Optional[str]` \| `None` \| Language hint \|
	369	+\| `speaker_hints` \| `Optional[list[str]]` \| `None` \| Speaker names for better recognition \|
	370	+
	371	+Returns: `dict` -- transcription result with `text`, `segments`, `duration`.
	372	+
	373	+Local Whisper: If `transcription_model` is unset or starts with `"whisper-local"`, the manager tries local Whisper first. Use `"whisper-local:large"` to specify a model size.
	374	+
	375	+### get_models_used()
	376	+
	377	+```python
	378	+def get_models_used(self) -> dict[str, str]
	379	+```
	380	+
	381	+Return a dict mapping capability to `"provider/model"` string for tracking purposes.
	382	+
	383	+```python
	384	+pm = ProviderManager()
	385	+print(pm.get_models_used())
	386	+# {"vision": "gemini/gemini-2.5-flash", "chat": "anthropic/claude-haiku-4-5-20251001", ...}
	387	+```
	388	+
	389	+### Usage examples
	390	+
	391	+```python
	392	+from video_processor.providers.manager import ProviderManager
	393	+
	394	+# Auto-select best providers
	395	+pm = ProviderManager()
	396	+
	397	+# Force everything through one provider
	398	+pm = ProviderManager(provider="openai")
	399	+
	400	+# Explicit model selection
	401	+pm = ProviderManager(
	402	+ vision_model="gpt-4o",
	403	+ chat_model="claude-haiku-4-5-20251001",
	404	+ transcription_model="whisper-local:large",
	405	+)
	406	+
	407	+# Chat completion
	408	+response = pm.chat([
	409	+ {"role": "user", "content": "Summarize this meeting transcript..."}
	410	+])
	411	+
	412	+# Image analysis
	413	+with open("diagram.png", "rb") as f:
	414	+ analysis = pm.analyze_image(f.read(), "Describe this architecture diagram")
	415	+
	416	+# Transcription with speaker hints
	417	+result = pm.transcribe_audio(
	418	+ "meeting.mp3",
	419	+ language="en",
	420	+ speaker_hints=["Alice", "Bob", "Charlie"],
	421	+)
	422	+
	423	+# Check usage
	424	+print(pm.usage.summary())
	425	+```
	426	+
	427	+---
	428	+
	429	+## discover_available_models()
	430	+
	431	+```python
	432	+from video_processor.providers.discovery import discover_available_models
	433	+```
	434	+
	435	+```python
	436	+def discover_available_models(
	437	+ api_keys: Optional[dict[str, str]] = None,
	438	+ force_refresh: bool = False,
	439	+) -> list[ModelInfo]
	440	+```
	441	+
	442	+Discover available models from all configured providers. For each provider with a valid API key, calls `list_models()` and returns a unified, sorted list.
	443	+
	444	+Parameters:
	445	+
	446	+\| Parameter \| Type \| Default \| Description \|
	447	+\|---\|---\|---\|---\|
	448	+\| `api_keys` \| `Optional[dict[str, str]]` \| `None` \| Override API keys (defaults to environment variables) \|
	449	+\| `force_refresh` \| `bool` \| `False` \| Force re-discovery, ignoring the session cache \|
	450	+
	451	+Returns: `list[ModelInfo]` -- all discovered models, sorted by provider then model ID.
	452	+
	453	+Caching: Results are cached for the session. Use `force_refresh=True` or `clear_discovery_cache()` to refresh.
	454	+
	455	+```python
	456	+from video_processor.providers.discovery import (
	457	+ discover_available_models,
	458	+ clear_discovery_cache,
	459	+)
	460	+
	461	+# Discover models using environment variables
	462	+models = discover_available_models()
	463	+for m in models:
	464	+ print(f"{m.provider}/{m.id} - {m.capabilities}")
	465	+
	466	+# Force refresh
	467	+models = discover_available_models(force_refresh=True)
	468	+
	469	+# Override API keys
	470	+models = discover_available_models(api_keys={
	471	+ "openai": "sk-...",
	472	+ "anthropic": "sk-ant-...",
	473	+})
	474	+
	475	+# Clear cache
	476	+clear_discovery_cache()
	477	+```
	478	+
	479	+### clear_discovery_cache()
	480	+
	481	+```python
	482	+def clear_discovery_cache() -> None
	483	+```
	484	+
	485	+Clear the cached model list, forcing the next `discover_available_models()` call to re-query providers.
	486	+
	487	+---
	488	+
	489	+## Built-in Providers
	490	+
	491	+The following providers are registered automatically when the provider system initializes:
	492	+
	493	+\| Provider \| Environment Variable \| Capabilities \| Default Chat Model \|
	494	+\|---\|---\|---\|---\|
	495	+\| `openai` \| `OPENAI_API_KEY` \| chat, vision, audio \| `gpt-4o-mini` \|
	496	+\| `anthropic` \| `ANTHROPIC_API_KEY` \| chat, vision \| `claude-haiku-4-5-20251001` \|
	497	+\| `gemini` \| `GEMINI_API_KEY` \| chat, vision, audio \| `gemini-2.5-flash` \|
	498	+\| `ollama` \| (none -- checks server) \| chat, vision \| (depends on installed models) \|
	499	+\| `together` \| `TOGETHER_API_KEY` \| chat \| (varies) \|
	500	+\| `fireworks` \| `FIREWORKS_API_KEY` \| chat \| (varies) \|
	501	+\| `cerebras` \| `CEREBRAS_API_KEY` \| chat \| (varies) \|
	502	+\| `xai` \| `XAI_API_KEY` \| chat \| (varies) \|
	503	+\| `azure` \| `AZURE_OPENAI_API_KEY` \| chat, vision \| (varies) \|
8	504
9	505	ADDED docs/api/sources.md

	--- docs/api/providers.md
	+++ docs/api/providers.md
	@@ -3,5 +3,501 @@
3	::: video_processor.providers.base
4
5	::: video_processor.providers.manager
6
7	::: video_processor.providers.discovery
















































































































































































































































































































































































































































































































8
9	DDED docs/api/sources.md

	--- docs/api/providers.md
	+++ docs/api/providers.md
	@@ -3,5 +3,501 @@
3	::: video_processor.providers.base
4
5	::: video_processor.providers.manager
6
7	::: video_processor.providers.discovery
8
9	---
10
11	## Overview
12
13	The provider system abstracts LLM API calls behind a unified interface. It supports multiple providers (OpenAI, Anthropic, Gemini, Ollama, and OpenAI-compatible services), automatic model discovery, capability-based routing, and usage tracking.
14
15	Key components:
16
17	- `BaseProvider` -- abstract interface that all providers implement
18	- `ProviderRegistry` -- global registry mapping provider names to classes
19	- `ProviderManager` -- high-level router that picks the best provider for each task
20	- `discover_available_models()` -- scans all configured providers for available models
21
22	---
23
24	## BaseProvider (ABC)
25
26	```python
27	from video_processor.providers.base import BaseProvider
28	```
29
30	Abstract base class that all provider implementations must subclass. Defines the four core capabilities: chat, vision, audio transcription, and model listing.
31
32	Class attribute:
33
34	\| Attribute \| Type \| Description \|
35	\|---\|---\|---\|
36	\| `provider_name` \| `str` \| Identifier for this provider (e.g., `"openai"`, `"anthropic"`) \|
37
38	### chat()
39
40	```python
41	def chat(
42	self,
43	messages: list[dict],
44	max_tokens: int = 4096,
45	temperature: float = 0.7,
46	model: Optional[str] = None,
47	) -> str
48	```
49
50	Send a chat completion request.
51
52	Parameters:
53
54	\| Parameter \| Type \| Default \| Description \|
55	\|---\|---\|---\|---\|
56	\| `messages` \| `list[dict]` \| required \| OpenAI-format message list (`role`, `content`) \|
57	\| `max_tokens` \| `int` \| `4096` \| Maximum tokens in the response \|
58	\| `temperature` \| `float` \| `0.7` \| Sampling temperature \|
59	\| `model` \| `Optional[str]` \| `None` \| Override model ID \|
60
61	Returns: `str` -- the assistant's text response.
62
63	### analyze_image()
64
65	```python
66	def analyze_image(
67	self,
68	image_bytes: bytes,
69	prompt: str,
70	max_tokens: int = 4096,
71	model: Optional[str] = None,
72	) -> str
73	```
74
75	Analyze an image with a text prompt using a vision-capable model.
76
77	Parameters:
78
79	\| Parameter \| Type \| Default \| Description \|
80	\|---\|---\|---\|---\|
81	\| `image_bytes` \| `bytes` \| required \| Raw image data (JPEG, PNG, etc.) \|
82	\| `prompt` \| `str` \| required \| Analysis instructions \|
83	\| `max_tokens` \| `int` \| `4096` \| Maximum tokens in the response \|
84	\| `model` \| `Optional[str]` \| `None` \| Override model ID \|
85
86	Returns: `str` -- the assistant's analysis text.
87
88	### transcribe_audio()
89
90	```python
91	def transcribe_audio(
92	self,
93	audio_path: str \| Path,
94	language: Optional[str] = None,
95	model: Optional[str] = None,
96	) -> dict
97	```
98
99	Transcribe an audio file.
100
101	Parameters:
102
103	\| Parameter \| Type \| Default \| Description \|
104	\|---\|---\|---\|---\|
105	\| `audio_path` \| `str \\| Path` \| required \| Path to the audio file \|
106	\| `language` \| `Optional[str]` \| `None` \| Language hint (ISO 639-1 code) \|
107	\| `model` \| `Optional[str]` \| `None` \| Override model ID \|
108
109	Returns: `dict` -- transcription result with keys `text`, `segments`, `duration`, etc.
110
111	### list_models()
112
113	```python
114	def list_models(self) -> list[ModelInfo]
115	```
116
117	Discover available models from this provider's API.
118
119	Returns: `list[ModelInfo]` -- available models with capability metadata.
120
121	---
122
123	## ModelInfo
124
125	```python
126	from video_processor.providers.base import ModelInfo
127	```
128
129	Pydantic model describing an available model from a provider.
130
131	\| Field \| Type \| Default \| Description \|
132	\|---\|---\|---\|---\|
133	\| `id` \| `str` \| required \| Model identifier (e.g., `"gpt-4o"`, `"claude-haiku-4-5-20251001"`) \|
134	\| `provider` \| `str` \| required \| Provider name (e.g., `"openai"`, `"anthropic"`, `"gemini"`) \|
135	\| `display_name` \| `str` \| `""` \| Human-readable display name \|
136	\| `capabilities` \| `List[str]` \| `[]` \| Model capabilities: `"chat"`, `"vision"`, `"audio"`, `"embedding"` \|
137
138	```json
139	{
140	"id": "gpt-4o",
141	"provider": "openai",
142	"display_name": "GPT-4o",
143	"capabilities": ["chat", "vision"]
144	}
145	```
146
147	---
148
149	## ProviderRegistry
150
151	```python
152	from video_processor.providers.base import ProviderRegistry
153	```
154
155	Class-level registry for provider classes. Providers register themselves with metadata on import. This registry is used internally by `ProviderManager` but can also be used directly for introspection.
156
157	### register()
158
159	```python
160	@classmethod
161	def register(
162	cls,
163	name: str,
164	provider_class: type,
165	env_var: str = "",
166	model_prefixes: Optional[List[str]] = None,
167	default_models: Optional[Dict[str, str]] = None,
168	) -> None
169	```
170
171	Register a provider class with its metadata. Called by each provider module at import time.
172
173	Parameters:
174
175	\| Parameter \| Type \| Default \| Description \|
176	\|---\|---\|---\|---\|
177	\| `name` \| `str` \| required \| Provider name (e.g., `"openai"`) \|
178	\| `provider_class` \| `type` \| required \| The provider class \|
179	\| `env_var` \| `str` \| `""` \| Environment variable for API key \|
180	\| `model_prefixes` \| `Optional[List[str]]` \| `None` \| Model ID prefixes for auto-detection (e.g., `["gpt-", "o1-"]`) \|
181	\| `default_models` \| `Optional[Dict[str, str]]` \| `None` \| Default models per capability (e.g., `{"chat": "gpt-4o", "vision": "gpt-4o"}`) \|
182
183	### get()
184
185	```python
186	@classmethod
187	def get(cls, name: str) -> type
188	```
189
190	Return the provider class for a given name. Raises `ValueError` if the provider is not registered.
191
192	### get_by_model()
193
194	```python
195	@classmethod
196	def get_by_model(cls, model_id: str) -> Optional[str]
197	```
198
199	Return the provider name for a model ID based on prefix matching. Returns `None` if no match is found.
200
201	### get_default_models()
202
203	```python
204	@classmethod
205	def get_default_models(cls, name: str) -> Dict[str, str]
206	```
207
208	Return the default models dict for a provider, mapping capability names to model IDs.
209
210	### available()
211
212	```python
213	@classmethod
214	def available(cls) -> List[str]
215	```
216
217	Return names of providers whose required environment variable is set (or providers with no env var requirement, like Ollama).
218
219	### all_registered()
220
221	```python
222	@classmethod
223	def all_registered(cls) -> Dict[str, Dict]
224	```
225
226	Return all registered providers and their metadata dictionaries.
227
228	---
229
230	## OpenAICompatibleProvider
231
232	```python
233	from video_processor.providers.base import OpenAICompatibleProvider
234	```
235
236	Base class for providers using OpenAI-compatible APIs (Together, Fireworks, Cerebras, xAI, Azure). Implements `chat()`, `analyze_image()`, and `list_models()` using the OpenAI client library. `transcribe_audio()` raises `NotImplementedError` by default.
237
238	Constructor:
239
240	```python
241	def __init__(self, api_key: Optional[str] = None, base_url: Optional[str] = None)
242	```
243
244	\| Parameter \| Type \| Default \| Description \|
245	\|---\|---\|---\|---\|
246	\| `api_key` \| `Optional[str]` \| `None` \| API key (falls back to `self.env_var` environment variable) \|
247	\| `base_url` \| `Optional[str]` \| `None` \| API base URL (falls back to `self.base_url` class attribute) \|
248
249	Subclass attributes to override:
250
251	\| Attribute \| Description \|
252	\|---\|---\|
253	\| `provider_name` \| Provider identifier string \|
254	\| `base_url` \| Default API base URL \|
255	\| `env_var` \| Environment variable name for the API key \|
256
257	Usage tracking: After each `chat()` or `analyze_image()` call, the provider stores token counts in `self._last_usage` as `{"input_tokens": int, "output_tokens": int}`. This is consumed by `ProviderManager._track()`.
258
259	---
260
261	## ProviderManager
262
263	```python
264	from video_processor.providers.manager import ProviderManager
265	```
266
267	High-level router that selects the best available provider and model for each API call. Supports explicit model selection, forced provider, or automatic selection based on discovered capabilities.
268
269	### Constructor
270
271	```python
272	def __init__(
273	self,
274	vision_model: Optional[str] = None,
275	chat_model: Optional[str] = None,
276	transcription_model: Optional[str] = None,
277	provider: Optional[str] = None,
278	auto: bool = True,
279	)
280	```
281
282	\| Parameter \| Type \| Default \| Description \|
283	\|---\|---\|---\|---\|
284	\| `vision_model` \| `Optional[str]` \| `None` \| Override model for vision tasks (e.g., `"gpt-4o"`) \|
285	\| `chat_model` \| `Optional[str]` \| `None` \| Override model for chat/LLM tasks \|
286	\| `transcription_model` \| `Optional[str]` \| `None` \| Override model for transcription \|
287	\| `provider` \| `Optional[str]` \| `None` \| Force all tasks to a single provider \|
288	\| `auto` \| `bool` \| `True` \| If `True` and no model specified, pick the best available \|
289
290	Attributes:
291
292	\| Attribute \| Type \| Description \|
293	\|---\|---\|---\|
294	\| `usage` \| `UsageTracker` \| Tracks token counts and API costs across all calls \|
295
296	### Auto-selection preferences
297
298	When `auto=True` and no explicit model is set, providers are tried in this order:
299
300	Vision: Gemini (`gemini-2.5-flash`) > OpenAI (`gpt-4o-mini`) > Anthropic (`claude-haiku-4-5-20251001`)
301
302	Chat: Anthropic (`claude-haiku-4-5-20251001`) > OpenAI (`gpt-4o-mini`) > Gemini (`gemini-2.5-flash`)
303
304	Transcription: OpenAI (`whisper-1`) > Gemini (`gemini-2.5-flash`)
305
306	If no API-key-based provider is available, Ollama is tried as a fallback.
307
308	### chat()
309
310	```python
311	def chat(
312	self,
313	messages: list[dict],
314	max_tokens: int = 4096,
315	temperature: float = 0.7,
316	) -> str
317	```
318
319	Send a chat completion to the best available provider. Automatically resolves which provider and model to use.
320
321	Parameters:
322
323	\| Parameter \| Type \| Default \| Description \|
324	\|---\|---\|---\|---\|
325	\| `messages` \| `list[dict]` \| required \| OpenAI-format messages \|
326	\| `max_tokens` \| `int` \| `4096` \| Maximum response tokens \|
327	\| `temperature` \| `float` \| `0.7` \| Sampling temperature \|
328
329	Returns: `str` -- assistant response text.
330
331	Raises: `RuntimeError` if no provider is available for the `chat` capability.
332
333	### analyze_image()
334
335	```python
336	def analyze_image(
337	self,
338	image_bytes: bytes,
339	prompt: str,
340	max_tokens: int = 4096,
341	) -> str
342	```
343
344	Analyze an image using the best available vision provider.
345
346	Returns: `str` -- analysis text.
347
348	Raises: `RuntimeError` if no provider is available for the `vision` capability.
349
350	### transcribe_audio()
351
352	```python
353	def transcribe_audio(
354	self,
355	audio_path: str \| Path,
356	language: Optional[str] = None,
357	speaker_hints: Optional[list[str]] = None,
358	) -> dict
359	```
360
361	Transcribe audio. Prefers local Whisper (no file size limits, no API costs) when available, falling back to API-based transcription.
362
363	Parameters:
364
365	\| Parameter \| Type \| Default \| Description \|
366	\|---\|---\|---\|---\|
367	\| `audio_path` \| `str \\| Path` \| required \| Path to the audio file \|
368	\| `language` \| `Optional[str]` \| `None` \| Language hint \|
369	\| `speaker_hints` \| `Optional[list[str]]` \| `None` \| Speaker names for better recognition \|
370
371	Returns: `dict` -- transcription result with `text`, `segments`, `duration`.
372
373	Local Whisper: If `transcription_model` is unset or starts with `"whisper-local"`, the manager tries local Whisper first. Use `"whisper-local:large"` to specify a model size.
374
375	### get_models_used()
376
377	```python
378	def get_models_used(self) -> dict[str, str]
379	```
380
381	Return a dict mapping capability to `"provider/model"` string for tracking purposes.
382
383	```python
384	pm = ProviderManager()
385	print(pm.get_models_used())
386	# {"vision": "gemini/gemini-2.5-flash", "chat": "anthropic/claude-haiku-4-5-20251001", ...}
387	```
388
389	### Usage examples
390
391	```python
392	from video_processor.providers.manager import ProviderManager
393
394	# Auto-select best providers
395	pm = ProviderManager()
396
397	# Force everything through one provider
398	pm = ProviderManager(provider="openai")
399
400	# Explicit model selection
401	pm = ProviderManager(
402	vision_model="gpt-4o",
403	chat_model="claude-haiku-4-5-20251001",
404	transcription_model="whisper-local:large",
405	)
406
407	# Chat completion
408	response = pm.chat([
409	{"role": "user", "content": "Summarize this meeting transcript..."}
410	])
411
412	# Image analysis
413	with open("diagram.png", "rb") as f:
414	analysis = pm.analyze_image(f.read(), "Describe this architecture diagram")
415
416	# Transcription with speaker hints
417	result = pm.transcribe_audio(
418	"meeting.mp3",
419	language="en",
420	speaker_hints=["Alice", "Bob", "Charlie"],
421	)
422
423	# Check usage
424	print(pm.usage.summary())
425	```
426
427	---
428
429	## discover_available_models()
430
431	```python
432	from video_processor.providers.discovery import discover_available_models
433	```
434
435	```python
436	def discover_available_models(
437	api_keys: Optional[dict[str, str]] = None,
438	force_refresh: bool = False,
439	) -> list[ModelInfo]
440	```
441
442	Discover available models from all configured providers. For each provider with a valid API key, calls `list_models()` and returns a unified, sorted list.
443
444	Parameters:
445
446	\| Parameter \| Type \| Default \| Description \|
447	\|---\|---\|---\|---\|
448	\| `api_keys` \| `Optional[dict[str, str]]` \| `None` \| Override API keys (defaults to environment variables) \|
449	\| `force_refresh` \| `bool` \| `False` \| Force re-discovery, ignoring the session cache \|
450
451	Returns: `list[ModelInfo]` -- all discovered models, sorted by provider then model ID.
452
453	Caching: Results are cached for the session. Use `force_refresh=True` or `clear_discovery_cache()` to refresh.
454
455	```python
456	from video_processor.providers.discovery import (
457	discover_available_models,
458	clear_discovery_cache,
459	)
460
461	# Discover models using environment variables
462	models = discover_available_models()
463	for m in models:
464	print(f"{m.provider}/{m.id} - {m.capabilities}")
465
466	# Force refresh
467	models = discover_available_models(force_refresh=True)
468
469	# Override API keys
470	models = discover_available_models(api_keys={
471	"openai": "sk-...",
472	"anthropic": "sk-ant-...",
473	})
474
475	# Clear cache
476	clear_discovery_cache()
477	```
478
479	### clear_discovery_cache()
480
481	```python
482	def clear_discovery_cache() -> None
483	```
484
485	Clear the cached model list, forcing the next `discover_available_models()` call to re-query providers.
486
487	---
488
489	## Built-in Providers
490
491	The following providers are registered automatically when the provider system initializes:
492
493	\| Provider \| Environment Variable \| Capabilities \| Default Chat Model \|
494	\|---\|---\|---\|---\|
495	\| `openai` \| `OPENAI_API_KEY` \| chat, vision, audio \| `gpt-4o-mini` \|
496	\| `anthropic` \| `ANTHROPIC_API_KEY` \| chat, vision \| `claude-haiku-4-5-20251001` \|
497	\| `gemini` \| `GEMINI_API_KEY` \| chat, vision, audio \| `gemini-2.5-flash` \|
498	\| `ollama` \| (none -- checks server) \| chat, vision \| (depends on installed models) \|
499	\| `together` \| `TOGETHER_API_KEY` \| chat \| (varies) \|
500	\| `fireworks` \| `FIREWORKS_API_KEY` \| chat \| (varies) \|
501	\| `cerebras` \| `CEREBRAS_API_KEY` \| chat \| (varies) \|
502	\| `xai` \| `XAI_API_KEY` \| chat \| (varies) \|
503	\| `azure` \| `AZURE_OPENAI_API_KEY` \| chat, vision \| (varies) \|
504
505	DDED docs/api/sources.md

A docs/api/sources.md

+281

		--- a/docs/api/sources.md
		+++ b/docs/api/sources.md
		@@ -0,0 +1,281 @@
	1	+# Sources API Reference
	2	+
	3	+::: video_processor.sources.base
	4	+
	5	+---
	6	+
	7	+## Overview
	8	+
	9	+The sources module provides a unified interface for fetching content from cloud services, local applications, and the web. All sources implement the `BaseSource` abstract class, providing consistent `authenticate()`, `list_videos()`, and `download()` methods.
	10	+
	11	+Sources are lazy-loaded to avoid pulling in optional dependencies at import time. You can import any source directly from `video_processor.sources` and the correct module will be loaded on demand.
	12	+
	13	+---
	14	+
	15	+## BaseSource (ABC)
	16	+
	17	+```python
	18	+from video_processor.sources import BaseSource
	19	+```
	20	+
	21	+Abstract base class that all source integrations implement. Defines the standard three-step workflow: authenticate, list, download.
	22	+
	23	+### authenticate()
	24	+
	25	+```python
	26	+@abstractmethod
	27	+def authenticate(self) -> bool
	28	+```
	29	+
	30	+Authenticate with the cloud provider or service. Uses the auth strategy defined for the source (OAuth, API key, local access, etc.).
	31	+
	32	+Returns: `bool` -- `True` on successful authentication, `False` on failure.
	33	+
	34	+### list_videos()
	35	+
	36	+```python
	37	+@abstractmethod
	38	+def list_videos(
	39	+ self,
	40	+ folder_id: Optional[str] = None,
	41	+ folder_path: Optional[str] = None,
	42	+ patterns: Optional[List[str]] = None,
	43	+) -> List[SourceFile]
	44	+```
	45	+
	46	+List available video files (or other content, depending on the source).
	47	+
	48	+Parameters:
	49	+
	50	+\| Parameter \| Type \| Default \| Description \|
	51	+\|---\|---\|---\|---\|
	52	+\| `folder_id` \| `Optional[str]` \| `None` \| Provider-specific folder/container identifier \|
	53	+\| `folder_path` \| `Optional[str]` \| `None` \| Path within the source (e.g., folder name) \|
	54	+\| `patterns` \| `Optional[List[str]]` \| `None` \| File name glob patterns to filter results \|
	55	+
	56	+Returns: `List[SourceFile]` -- available files matching the criteria.
	57	+
	58	+### download()
	59	+
	60	+```python
	61	+@abstractmethod
	62	+def download(
	63	+ self,
	64	+ file: SourceFile,
	65	+ destination: Path,
	66	+) -> Path
	67	+```
	68	+
	69	+Download a single file to a local path.
	70	+
	71	+Parameters:
	72	+
	73	+\| Parameter \| Type \| Description \|
	74	+\|---\|---\|---\|
	75	+\| `file` \| `SourceFile` \| File descriptor from `list_videos()` \|
	76	+\| `destination` \| `Path` \| Local destination path \|
	77	+
	78	+Returns: `Path` -- the local path where the file was saved.
	79	+
	80	+### download_all()
	81	+
	82	+```python
	83	+def download_all(
	84	+ self,
	85	+ files: List[SourceFile],
	86	+ destination_dir: Path,
	87	+) -> List[Path]
	88	+```
	89	+
	90	+Download multiple files to a directory, preserving subfolder structure from `SourceFile.path`. This is a concrete method provided by the base class.
	91	+
	92	+Parameters:
	93	+
	94	+\| Parameter \| Type \| Description \|
	95	+\|---\|---\|---\|
	96	+\| `files` \| `List[SourceFile]` \| Files to download \|
	97	+\| `destination_dir` \| `Path` \| Base directory for downloads (created if needed) \|
	98	+
	99	+Returns: `List[Path]` -- local paths of successfully downloaded files. Failed downloads are logged and skipped.
	100	+
	101	+---
	102	+
	103	+## SourceFile
	104	+
	105	+```python
	106	+from video_processor.sources import SourceFile
	107	+```
	108	+
	109	+Pydantic model describing a file available in a cloud source.
	110	+
	111	+\| Field \| Type \| Default \| Description \|
	112	+\|---\|---\|---\|---\|
	113	+\| `name` \| `str` \| required \| File name \|
	114	+\| `id` \| `str` \| required \| Provider-specific file identifier \|
	115	+\| `size_bytes` \| `Optional[int]` \| `None` \| File size in bytes \|
	116	+\| `mime_type` \| `Optional[str]` \| `None` \| MIME type (e.g., `"video/mp4"`) \|
	117	+\| `modified_at` \| `Optional[str]` \| `None` \| Last modified timestamp \|
	118	+\| `path` \| `Optional[str]` \| `None` \| Path within the source folder (used for subfolder structure in `download_all`) \|
	119	+
	120	+```json
	121	+{
	122	+ "name": "sprint-review-2026-03-01.mp4",
	123	+ "id": "abc123def456",
	124	+ "size_bytes": 524288000,
	125	+ "mime_type": "video/mp4",
	126	+ "modified_at": "2026-03-01T14:30:00Z",
	127	+ "path": "recordings/march/sprint-review-2026-03-01.mp4"
	128	+}
	129	+```
	130	+
	131	+---
	132	+
	133	+## Lazy Loading Pattern
	134	+
	135	+All sources are lazy-loaded via `__getattr__` in the package `__init__.py`. This means importing `video_processor.sources` does not pull in any external dependencies (e.g., `google-auth`, `msal`, `notion-client`). The actual module is loaded only when you access the class.
	136	+
	137	+```python
	138	+# This import is instant -- no dependencies loaded
	139	+from video_processor.sources import ZoomSource
	140	+
	141	+# The zoom_source module (and its dependencies) are loaded here
	142	+source = ZoomSource()
	143	+```
	144	+
	145	+---
	146	+
	147	+## Available Sources
	148	+
	149	+### Cloud Recordings
	150	+
	151	+Sources for fetching recorded meetings from video conferencing platforms.
	152	+
	153	+\| Source \| Class \| Auth Method \| Description \|
	154	+\|---\|---\|---\|---\|
	155	+\| Zoom \| `ZoomSource` \| OAuth / Server-to-Server \| List and download Zoom cloud recordings \|
	156	+\| Google Meet \| `MeetRecordingSource` \| OAuth (Google) \| List and download Google Meet recordings from Drive \|
	157	+\| Microsoft Teams \| `TeamsRecordingSource` \| OAuth (Microsoft) \| List and download Teams meeting recordings \|
	158	+
	159	+### Cloud Storage and Workspace
	160	+
	161	+Sources for accessing files stored in cloud platforms.
	162	+
	163	+\| Source \| Class \| Auth Method \| Description \|
	164	+\|---\|---\|---\|---\|
	165	+\| Google Drive \| `GoogleDriveSource` \| OAuth (Google) \| Files from Google Drive \|
	166	+\| Google Workspace \| `GWSSource` \| OAuth (Google) \| Google Docs, Sheets, Slides \|
	167	+\| Microsoft 365 \| `M365Source` \| OAuth (Microsoft) \| OneDrive, SharePoint files \|
	168	+\| Notion \| `NotionSource` \| OAuth / API key \| Notion pages and databases \|
	169	+\| GitHub \| `GitHubSource` \| OAuth / API token \| Repository files, issues, discussions \|
	170	+\| Dropbox \| `DropboxSource` \| OAuth / access token \| (via auth config) \|
	171	+
	172	+### Notes Applications
	173	+
	174	+Sources for local and cloud-based note-taking apps.
	175	+
	176	+\| Source \| Class \| Auth Method \| Description \|
	177	+\|---\|---\|---\|---\|
	178	+\| Apple Notes \| `AppleNotesSource` \| Local (macOS) \| Notes from Apple Notes.app \|
	179	+\| Obsidian \| `ObsidianSource` \| Local filesystem \| Markdown files from Obsidian vaults \|
	180	+\| Logseq \| `LogseqSource` \| Local filesystem \| Pages from Logseq graphs \|
	181	+\| OneNote \| `OneNoteSource` \| OAuth (Microsoft) \| Microsoft OneNote notebooks \|
	182	+\| Google Keep \| `GoogleKeepSource` \| OAuth (Google) \| Google Keep notes \|
	183	+
	184	+### Web and Content
	185	+
	186	+Sources for fetching content from the web.
	187	+
	188	+\| Source \| Class \| Auth Method \| Description \|
	189	+\|---\|---\|---\|---\|
	190	+\| YouTube \| `YouTubeSource` \| API key / OAuth \| YouTube video metadata and transcripts \|
	191	+\| Web \| `WebSource` \| None \| General web page content extraction \|
	192	+\| RSS \| `RSSSource` \| None \| RSS/Atom feed entries \|
	193	+\| Podcast \| `PodcastSource` \| None \| Podcast episodes from RSS feeds \|
	194	+\| arXiv \| `ArxivSource` \| None \| Academic papers from arXiv \|
	195	+\| Hacker News \| `HackerNewsSource` \| None \| Hacker News posts and comments \|
	196	+\| Reddit \| `RedditSource` \| API credentials \| Reddit posts and comments \|
	197	+\| Twitter/X \| `TwitterSource` \| API credentials \| Tweets and threads \|
	198	+
	199	+---
	200	+
	201	+## Auth Integration
	202	+
	203	+Most sources use PlanOpticon's unified auth system (see [Auth API](auth.md)). The typical pattern within a source implementation:
	204	+
	205	+```python
	206	+from video_processor.auth import get_auth_manager
	207	+
	208	+class MySource(BaseSource):
	209	+ def __init__(self):
	210	+ self._token = None
	211	+
	212	+ def authenticate(self) -> bool:
	213	+ manager = get_auth_manager("my_service")
	214	+ if manager:
	215	+ token = manager.get_token()
	216	+ if token:
	217	+ self._token = token
	218	+ return True
	219	+ return False
	220	+
	221	+ def list_videos(self, **kwargs) -> list[SourceFile]:
	222	+ if not self._token:
	223	+ raise RuntimeError("Not authenticated. Call authenticate() first.")
	224	+ # Use self._token to call the API
	225	+ ...
	226	+```
	227	+
	228	+---
	229	+
	230	+## Usage Examples
	231	+
	232	+### Listing and downloading Zoom recordings
	233	+
	234	+```python
	235	+from pathlib import Path
	236	+from video_processor.sources import ZoomSource
	237	+
	238	+source = ZoomSource()
	239	+if source.authenticate():
	240	+ recordings = source.list_videos()
	241	+ for rec in recordings:
	242	+ print(f"{rec.name} ({rec.size_bytes} bytes)")
	243	+
	244	+ # Download all to a local directory
	245	+ paths = source.download_all(recordings, Path("./downloads"))
	246	+```
	247	+
	248	+### Fetching from multiple sources
	249	+
	250	+```python
	251	+from pathlib import Path
	252	+from video_processor.sources import GoogleDriveSource, NotionSource
	253	+
	254	+# Google Drive
	255	+gdrive = GoogleDriveSource()
	256	+if gdrive.authenticate():
	257	+ files = gdrive.list_videos(
	258	+ folder_path="Meeting Recordings",
	259	+ patterns=[".mp4", ".webm"],
	260	+ )
	261	+ gdrive.download_all(files, Path("./drive-downloads"))
	262	+
	263	+# Notion
	264	+notion = NotionSource()
	265	+if notion.authenticate():
	266	+ pages = notion.list_videos() # Lists Notion pages
	267	+ for page in pages:
	268	+ print(f"Page: {page.name}")
	269	+```
	270	+
	271	+### YouTube content
	272	+
	273	+```python
	274	+from video_processor.sources import YouTubeSource
	275	+
	276	+yt = YouTubeSource()
	277	+if yt.authenticate():
	278	+ videos = yt.list_videos(folder_path="https://youtube.com/playlist?list=...")
	279	+ for v in videos:
	280	+ print(f"{v.name} - {v.id}")
	281	+```

	--- a/docs/api/sources.md
	+++ b/docs/api/sources.md
	@@ -0,0 +1,281 @@

	--- a/docs/api/sources.md
	+++ b/docs/api/sources.md
	@@ -0,0 +1,281 @@
1	# Sources API Reference
2
3	::: video_processor.sources.base
4
5	---
6
7	## Overview
8
9	The sources module provides a unified interface for fetching content from cloud services, local applications, and the web. All sources implement the `BaseSource` abstract class, providing consistent `authenticate()`, `list_videos()`, and `download()` methods.
10
11	Sources are lazy-loaded to avoid pulling in optional dependencies at import time. You can import any source directly from `video_processor.sources` and the correct module will be loaded on demand.
12
13	---
14
15	## BaseSource (ABC)
16
17	```python
18	from video_processor.sources import BaseSource
19	```
20
21	Abstract base class that all source integrations implement. Defines the standard three-step workflow: authenticate, list, download.
22
23	### authenticate()
24
25	```python
26	@abstractmethod
27	def authenticate(self) -> bool
28	```
29
30	Authenticate with the cloud provider or service. Uses the auth strategy defined for the source (OAuth, API key, local access, etc.).
31
32	Returns: `bool` -- `True` on successful authentication, `False` on failure.
33
34	### list_videos()
35
36	```python
37	@abstractmethod
38	def list_videos(
39	self,
40	folder_id: Optional[str] = None,
41	folder_path: Optional[str] = None,
42	patterns: Optional[List[str]] = None,
43	) -> List[SourceFile]
44	```
45
46	List available video files (or other content, depending on the source).
47
48	Parameters:
49
50	\| Parameter \| Type \| Default \| Description \|
51	\|---\|---\|---\|---\|
52	\| `folder_id` \| `Optional[str]` \| `None` \| Provider-specific folder/container identifier \|
53	\| `folder_path` \| `Optional[str]` \| `None` \| Path within the source (e.g., folder name) \|
54	\| `patterns` \| `Optional[List[str]]` \| `None` \| File name glob patterns to filter results \|
55
56	Returns: `List[SourceFile]` -- available files matching the criteria.
57
58	### download()
59
60	```python
61	@abstractmethod
62	def download(
63	self,
64	file: SourceFile,
65	destination: Path,
66	) -> Path
67	```
68
69	Download a single file to a local path.
70
71	Parameters:
72
73	\| Parameter \| Type \| Description \|
74	\|---\|---\|---\|
75	\| `file` \| `SourceFile` \| File descriptor from `list_videos()` \|
76	\| `destination` \| `Path` \| Local destination path \|
77
78	Returns: `Path` -- the local path where the file was saved.
79
80	### download_all()
81
82	```python
83	def download_all(
84	self,
85	files: List[SourceFile],
86	destination_dir: Path,
87	) -> List[Path]
88	```
89
90	Download multiple files to a directory, preserving subfolder structure from `SourceFile.path`. This is a concrete method provided by the base class.
91
92	Parameters:
93
94	\| Parameter \| Type \| Description \|
95	\|---\|---\|---\|
96	\| `files` \| `List[SourceFile]` \| Files to download \|
97	\| `destination_dir` \| `Path` \| Base directory for downloads (created if needed) \|
98
99	Returns: `List[Path]` -- local paths of successfully downloaded files. Failed downloads are logged and skipped.
100
101	---
102
103	## SourceFile
104
105	```python
106	from video_processor.sources import SourceFile
107	```
108
109	Pydantic model describing a file available in a cloud source.
110
111	\| Field \| Type \| Default \| Description \|
112	\|---\|---\|---\|---\|
113	\| `name` \| `str` \| required \| File name \|
114	\| `id` \| `str` \| required \| Provider-specific file identifier \|
115	\| `size_bytes` \| `Optional[int]` \| `None` \| File size in bytes \|
116	\| `mime_type` \| `Optional[str]` \| `None` \| MIME type (e.g., `"video/mp4"`) \|
117	\| `modified_at` \| `Optional[str]` \| `None` \| Last modified timestamp \|
118	\| `path` \| `Optional[str]` \| `None` \| Path within the source folder (used for subfolder structure in `download_all`) \|
119
120	```json
121	{
122	"name": "sprint-review-2026-03-01.mp4",
123	"id": "abc123def456",
124	"size_bytes": 524288000,
125	"mime_type": "video/mp4",
126	"modified_at": "2026-03-01T14:30:00Z",
127	"path": "recordings/march/sprint-review-2026-03-01.mp4"
128	}
129	```
130
131	---
132
133	## Lazy Loading Pattern
134
135	All sources are lazy-loaded via `__getattr__` in the package `__init__.py`. This means importing `video_processor.sources` does not pull in any external dependencies (e.g., `google-auth`, `msal`, `notion-client`). The actual module is loaded only when you access the class.
136
137	```python
138	# This import is instant -- no dependencies loaded
139	from video_processor.sources import ZoomSource
140
141	# The zoom_source module (and its dependencies) are loaded here
142	source = ZoomSource()
143	```
144
145	---
146
147	## Available Sources
148
149	### Cloud Recordings
150
151	Sources for fetching recorded meetings from video conferencing platforms.
152
153	\| Source \| Class \| Auth Method \| Description \|
154	\|---\|---\|---\|---\|
155	\| Zoom \| `ZoomSource` \| OAuth / Server-to-Server \| List and download Zoom cloud recordings \|
156	\| Google Meet \| `MeetRecordingSource` \| OAuth (Google) \| List and download Google Meet recordings from Drive \|
157	\| Microsoft Teams \| `TeamsRecordingSource` \| OAuth (Microsoft) \| List and download Teams meeting recordings \|
158
159	### Cloud Storage and Workspace
160
161	Sources for accessing files stored in cloud platforms.
162
163	\| Source \| Class \| Auth Method \| Description \|
164	\|---\|---\|---\|---\|
165	\| Google Drive \| `GoogleDriveSource` \| OAuth (Google) \| Files from Google Drive \|
166	\| Google Workspace \| `GWSSource` \| OAuth (Google) \| Google Docs, Sheets, Slides \|
167	\| Microsoft 365 \| `M365Source` \| OAuth (Microsoft) \| OneDrive, SharePoint files \|
168	\| Notion \| `NotionSource` \| OAuth / API key \| Notion pages and databases \|
169	\| GitHub \| `GitHubSource` \| OAuth / API token \| Repository files, issues, discussions \|
170	\| Dropbox \| `DropboxSource` \| OAuth / access token \| (via auth config) \|
171
172	### Notes Applications
173
174	Sources for local and cloud-based note-taking apps.
175
176	\| Source \| Class \| Auth Method \| Description \|
177	\|---\|---\|---\|---\|
178	\| Apple Notes \| `AppleNotesSource` \| Local (macOS) \| Notes from Apple Notes.app \|
179	\| Obsidian \| `ObsidianSource` \| Local filesystem \| Markdown files from Obsidian vaults \|
180	\| Logseq \| `LogseqSource` \| Local filesystem \| Pages from Logseq graphs \|
181	\| OneNote \| `OneNoteSource` \| OAuth (Microsoft) \| Microsoft OneNote notebooks \|
182	\| Google Keep \| `GoogleKeepSource` \| OAuth (Google) \| Google Keep notes \|
183
184	### Web and Content
185
186	Sources for fetching content from the web.
187
188	\| Source \| Class \| Auth Method \| Description \|
189	\|---\|---\|---\|---\|
190	\| YouTube \| `YouTubeSource` \| API key / OAuth \| YouTube video metadata and transcripts \|
191	\| Web \| `WebSource` \| None \| General web page content extraction \|
192	\| RSS \| `RSSSource` \| None \| RSS/Atom feed entries \|
193	\| Podcast \| `PodcastSource` \| None \| Podcast episodes from RSS feeds \|
194	\| arXiv \| `ArxivSource` \| None \| Academic papers from arXiv \|
195	\| Hacker News \| `HackerNewsSource` \| None \| Hacker News posts and comments \|
196	\| Reddit \| `RedditSource` \| API credentials \| Reddit posts and comments \|
197	\| Twitter/X \| `TwitterSource` \| API credentials \| Tweets and threads \|
198
199	---
200
201	## Auth Integration
202
203	Most sources use PlanOpticon's unified auth system (see [Auth API](auth.md)). The typical pattern within a source implementation:
204
205	```python
206	from video_processor.auth import get_auth_manager
207
208	class MySource(BaseSource):
209	def __init__(self):
210	self._token = None
211
212	def authenticate(self) -> bool:
213	manager = get_auth_manager("my_service")
214	if manager:
215	token = manager.get_token()
216	if token:
217	self._token = token
218	return True
219	return False
220
221	def list_videos(self, **kwargs) -> list[SourceFile]:
222	if not self._token:
223	raise RuntimeError("Not authenticated. Call authenticate() first.")
224	# Use self._token to call the API
225	...
226	```
227
228	---
229
230	## Usage Examples
231
232	### Listing and downloading Zoom recordings
233
234	```python
235	from pathlib import Path
236	from video_processor.sources import ZoomSource
237
238	source = ZoomSource()
239	if source.authenticate():
240	recordings = source.list_videos()
241	for rec in recordings:
242	print(f"{rec.name} ({rec.size_bytes} bytes)")
243
244	# Download all to a local directory
245	paths = source.download_all(recordings, Path("./downloads"))
246	```
247
248	### Fetching from multiple sources
249
250	```python
251	from pathlib import Path
252	from video_processor.sources import GoogleDriveSource, NotionSource
253
254	# Google Drive
255	gdrive = GoogleDriveSource()
256	if gdrive.authenticate():
257	files = gdrive.list_videos(
258	folder_path="Meeting Recordings",
259	patterns=[".mp4", ".webm"],
260	)
261	gdrive.download_all(files, Path("./drive-downloads"))
262
263	# Notion
264	notion = NotionSource()
265	if notion.authenticate():
266	pages = notion.list_videos() # Lists Notion pages
267	for page in pages:
268	print(f"Page: {page.name}")
269	```
270
271	### YouTube content
272
273	```python
274	from video_processor.sources import YouTubeSource
275
276	yt = YouTubeSource()
277	if yt.authenticate():
278	videos = yt.list_videos(folder_path="https://youtube.com/playlist?list=...")
279	for v in videos:
280	print(f"{v.name} - {v.id}")
281	```

M docs/architecture/pipeline.md

+292 -14

		--- docs/architecture/pipeline.md
		+++ docs/architecture/pipeline.md
		@@ -1,8 +1,14 @@
1	1	# Processing Pipeline
	2	+
	3	+PlanOpticon has four main pipelines: video analysis, document ingestion, source connector, and export. Each pipeline can operate independently, and they connect through the shared knowledge graph.
	4	+
	5	+---
2	6
3	7	## Single video pipeline
	8	+
	9	+The core video analysis pipeline processes a single video file through eight sequential steps with checkpoint/resume support.
4	10
5	11	```mermaid
6	12	sequenceDiagram
7	13	participant CLI
8	14	participant Pipeline
		@@ -9,49 +15,321 @@
9	15	participant FrameExtractor
10	16	participant AudioExtractor
11	17	participant Provider
12	18	participant DiagramAnalyzer
13	19	participant KnowledgeGraph
	20	+ participant Exporter
14	21
15	22	CLI->>Pipeline: process_single_video()
	23	+
	24	+ Note over Pipeline: Step 1: Extract frames
16	25	Pipeline->>FrameExtractor: extract_frames()
17	26	Note over FrameExtractor: Change detection + periodic capture (every 30s)
	27	+ FrameExtractor-->>Pipeline: frame_paths[]
	28	+
	29	+ Note over Pipeline: Step 2: Filter people frames
18	30	Pipeline->>Pipeline: filter_people_frames()
19	31	Note over Pipeline: OpenCV face detection removes webcam/people frames
	32	+
	33	+ Note over Pipeline: Step 3: Extract + transcribe audio
20	34	Pipeline->>AudioExtractor: extract_audio()
21	35	Pipeline->>Provider: transcribe_audio()
	36	+ Note over Provider: Supports speaker hints via --speakers flag
	37	+
	38	+ Note over Pipeline: Step 4: Analyze visuals
22	39	Pipeline->>DiagramAnalyzer: process_frames()
23		-
24		- loop Each frame
	40	+ loop Each frame (up to 10 standard / 20 comprehensive)
25	41	DiagramAnalyzer->>Provider: classify (vision)
26	42	alt High confidence diagram
27	43	DiagramAnalyzer->>Provider: full analysis
	44	+ Note over Provider: Extract description, text, mermaid, chart data
28	45	else Medium confidence
29	46	DiagramAnalyzer-->>Pipeline: screengrab fallback
30	47	end
31	48	end
32	49
	50	+ Note over Pipeline: Step 5: Build knowledge graph
	51	+ Pipeline->>KnowledgeGraph: register_source()
33	52	Pipeline->>KnowledgeGraph: process_transcript()
34	53	Pipeline->>KnowledgeGraph: process_diagrams()
	54	+ Note over KnowledgeGraph: Writes knowledge_graph.db (SQLite) + .json
	55	+
	56	+ Note over Pipeline: Step 6: Extract key points + action items
35	57	Pipeline->>Provider: extract key points
36	58	Pipeline->>Provider: extract action items
37		- Pipeline->>Pipeline: generate reports
38		- Pipeline->>Pipeline: export formats
	59	+
	60	+ Note over Pipeline: Step 7: Generate report
	61	+ Pipeline->>Pipeline: generate markdown report
	62	+ Note over Pipeline: Includes mermaid diagrams, tables, cross-references
	63	+
	64	+ Note over Pipeline: Step 8: Export formats
	65	+ Pipeline->>Exporter: export_all_formats()
	66	+ Note over Exporter: HTML report, PDF, SVG/PNG renderings, chart reproductions
	67	+
39	68	Pipeline-->>CLI: VideoManifest
40	69	```
	70	+
	71	+### Pipeline steps in detail
	72	+
	73	+\| Step \| Name \| Checkpointable \| Description \|
	74	+\|------\|------\|----------------\|-------------\|
	75	+\| 1 \| Extract frames \| Yes \| Change detection + periodic capture. Skipped if `frames/frame_*.jpg` exist on disk. \|
	76	+\| 2 \| Filter people frames \| No \| Inline with step 1. OpenCV face detection removes webcam frames. \|
	77	+\| 3 \| Extract + transcribe audio \| Yes \| Skipped if `transcript/transcript.json` exists. Speaker hints passed if `--speakers` provided. \|
	78	+\| 4 \| Analyze visuals \| Yes \| Skipped if `diagrams/` is populated. Evenly samples frames (not just first N). \|
	79	+\| 5 \| Build knowledge graph \| Yes \| Skipped if `results/knowledge_graph.db` exists. Registers source, processes transcript and diagrams. \|
	80	+\| 6 \| Extract key points + actions \| Yes \| Skipped if `results/key_points.json` and `results/action_items.json` exist. \|
	81	+\| 7 \| Generate report \| Yes \| Skipped if `results/analysis.md` exists. \|
	82	+\| 8 \| Export formats \| No \| Always runs. Renders mermaid to SVG/PNG, reproduces charts, generates HTML/PDF. \|
	83	+
	84	+---
41	85
42	86	## Batch pipeline
43	87
44		-The batch command wraps the single-video pipeline:
	88	+The batch pipeline wraps the single-video pipeline and adds cross-video knowledge graph merging.
	89	+
	90	+```mermaid
	91	+flowchart TD
	92	+ A[Scan input directory] --> B[Match video files by pattern]
	93	+ B --> C{For each video}
	94	+ C --> D[process_single_video]
	95	+ D --> E{Success?}
	96	+ E -->\|Yes\| F[Collect manifest + KG]
	97	+ E -->\|No\| G[Log error, continue]
	98	+ F --> H[Next video]
	99	+ G --> H
	100	+ H --> C
	101	+ C -->\|All done\| I[Merge knowledge graphs]
	102	+ I --> J[Fuzzy matching + conflict resolution]
	103	+ J --> K[Generate batch summary]
	104	+ K --> L[Write batch manifest]
	105	+ L --> M[batch_manifest.json + batch_summary.md + merged KG]
	106	+```
	107	+
	108	+### Knowledge graph merge strategy
	109	+
	110	+During batch merging, `KnowledgeGraph.merge()` applies:
	111	+
	112	+1. Case-insensitive exact matching for entity names
	113	+2. Fuzzy matching via `SequenceMatcher` (threshold >= 0.85) for near-duplicates
	114	+3. Type conflict resolution using a specificity ranking (e.g., `technology` > `concept`)
	115	+4. Description union across all sources
	116	+5. Relationship deduplication by (source, target, type) tuple
	117	+
	118	+---
	119	+
	120	+## Document ingestion pipeline
	121	+
	122	+The document ingestion pipeline processes files (Markdown, plaintext, PDF) into knowledge graphs without video analysis.
	123	+
	124	+```mermaid
	125	+flowchart TD
	126	+ A[Input: file or directory] --> B{File or directory?}
	127	+ B -->\|File\| C[get_processor by extension]
	128	+ B -->\|Directory\| D[Glob for supported extensions]
	129	+ D --> E{Recursive?}
	130	+ E -->\|Yes\| F[rglob all files]
	131	+ E -->\|No\| G[glob top-level only]
	132	+ F --> H[For each file]
	133	+ G --> H
	134	+ H --> C
	135	+ C --> I[DocumentProcessor.process]
	136	+ I --> J[DocumentChunk list]
	137	+ J --> K[Register source in KG]
	138	+ K --> L[Add chunks as content]
	139	+ L --> M[KG extracts entities + relationships]
	140	+ M --> N[knowledge_graph.db]
	141	+```
	142	+
	143	+### Supported document types
	144	+
	145	+\| Extension \| Processor \| Notes \|
	146	+\|-----------\|-----------\|-------\|
	147	+\| `.md` \| `MarkdownProcessor` \| Splits by headings into sections \|
	148	+\| `.txt` \| `PlaintextProcessor` \| Splits into fixed-size chunks \|
	149	+\| `.pdf` \| `PdfProcessor` \| Requires `pymupdf` or `pdfplumber`. Falls back gracefully between libraries. \|
	150	+
	151	+### Adding documents to an existing graph
	152	+
	153	+The `--db-path` flag lets you ingest documents into an existing knowledge graph:
	154	+
	155	+```bash
	156	+planopticon ingest spec.md --db-path existing.db
	157	+planopticon ingest ./docs/ -o ./output --recursive
	158	+```
	159	+
	160	+---
	161	+
	162	+## Source connector pipeline
	163	+
	164	+Source connectors fetch content from cloud services, note-taking apps, and web sources. Each source implements the `BaseSource` ABC with three methods: `authenticate()`, `list_videos()`, and `download()`.
	165	+
	166	+```mermaid
	167	+flowchart TD
	168	+ A[Source command] --> B[Authenticate with provider]
	169	+ B --> C{Auth success?}
	170	+ C -->\|No\| D[Error: check credentials]
	171	+ C -->\|Yes\| E[List files in folder]
	172	+ E --> F[Filter by pattern / type]
	173	+ F --> G[Download to local path]
	174	+ G --> H{Analyze or ingest?}
	175	+ H -->\|Video\| I[process_single_video / batch]
	176	+ H -->\|Document\| J[ingest_file / ingest_directory]
	177	+ I --> K[Knowledge graph]
	178	+ J --> K
	179	+```
	180	+
	181	+### Available sources
	182	+
	183	+PlanOpticon includes connectors for:
	184	+
	185	+\| Category \| Sources \|
	186	+\|----------\|---------\|
	187	+\| Cloud storage \| Google Drive, S3, Dropbox \|
	188	+\| Meeting recordings \| Zoom, Google Meet, Microsoft Teams \|
	189	+\| Productivity suites \| Google Workspace (Docs/Sheets/Slides), Microsoft 365 (SharePoint/OneDrive/OneNote) \|
	190	+\| Note-taking apps \| Obsidian, Logseq, Apple Notes, Google Keep, Notion \|
	191	+\| Web sources \| YouTube, Web (URL), RSS, Podcasts \|
	192	+\| Developer platforms \| GitHub, arXiv \|
	193	+\| Social media \| Reddit, Twitter/X, Hacker News \|
	194	+
	195	+Each source authenticates via environment variables (API keys, OAuth tokens) specific to the provider.
	196	+
	197	+---
	198	+
	199	+## Planning agent pipeline
	200	+
	201	+The planning agent consumes a knowledge graph and uses registered skills to generate planning artifacts.
	202	+
	203	+```mermaid
	204	+flowchart TD
	205	+ A[Knowledge graph] --> B[Load into AgentContext]
	206	+ B --> C[GraphQueryEngine]
	207	+ C --> D[Taxonomy classification]
	208	+ D --> E[Agent orchestrator]
	209	+ E --> F{Select skill}
	210	+ F --> G[ProjectPlan skill]
	211	+ F --> H[PRD skill]
	212	+ F --> I[Roadmap skill]
	213	+ F --> J[TaskBreakdown skill]
	214	+ F --> K[DocGenerator skill]
	215	+ F --> L[WikiGenerator skill]
	216	+ F --> M[NotesExport skill]
	217	+ F --> N[ArtifactExport skill]
	218	+ F --> O[GitHubIntegration skill]
	219	+ F --> P[RequirementsChat skill]
	220	+ G --> Q[Artifact output]
	221	+ H --> Q
	222	+ I --> Q
	223	+ J --> Q
	224	+ K --> Q
	225	+ L --> Q
	226	+ M --> Q
	227	+ N --> Q
	228	+ O --> Q
	229	+ P --> Q
	230	+ Q --> R[Write to disk / push to service]
	231	+```
	232	+
	233	+### Skill execution flow
	234	+
	235	+1. The `AgentContext` is populated with the knowledge graph, query engine, provider manager, and any planning entities from taxonomy classification
	236	+2. Each `Skill` checks `can_execute()` against the context (requires at minimum a knowledge graph and provider manager)
	237	+3. The skill's `execute()` method generates an `Artifact` with a name, content, type, and format
	238	+4. Artifacts are collected and can be exported to disk or pushed to external services (GitHub issues, wiki pages, etc.)
	239	+
	240	+---
	241	+
	242	+## Export pipeline
	243	+
	244	+The export pipeline converts knowledge graphs and analysis artifacts into various output formats.
	245	+
	246	+```mermaid
	247	+flowchart TD
	248	+ A[knowledge_graph.db] --> B{Export command}
	249	+ B --> C[export markdown]
	250	+ B --> D[export obsidian]
	251	+ B --> E[export notion]
	252	+ B --> F[export exchange]
	253	+ B --> G[wiki generate]
	254	+ B --> H[kg convert]
	255	+ C --> I[7 document types + entity briefs + CSV]
	256	+ D --> J[Obsidian vault with frontmatter + wiki-links]
	257	+ E --> K[Notion-compatible markdown + CSV database]
	258	+ F --> L[PlanOpticonExchange JSON payload]
	259	+ G --> M[GitHub wiki pages + sidebar + home]
	260	+ H --> N[Convert between .db / .json / .graphml / .csv]
	261	+```
	262	+
	263	+All export commands accept a `knowledge_graph.db` (or `.json`) path as input. No API key is required for template-based exports (markdown, obsidian, notion, wiki, exchange, convert). Only the planning agent skills that generate new content require a provider.
	264	+
	265	+---
	266	+
	267	+## How pipelines connect
	268	+
	269	+```mermaid
	270	+flowchart LR
	271	+ V[Video files] --> VP[Video Pipeline]
	272	+ D[Documents] --> DI[Document Ingestion]
	273	+ S[Cloud Sources] --> SC[Source Connectors]
	274	+ SC --> V
	275	+ SC --> D
	276	+ VP --> KG[(knowledge_graph.db)]
	277	+ DI --> KG
	278	+ KG --> QE[Query Engine]
	279	+ KG --> EP[Export Pipeline]
	280	+ KG --> PA[Planning Agent]
	281	+ PA --> AR[Artifacts]
	282	+ AR --> EP
	283	+```
	284	+
	285	+All pipelines converge on the knowledge graph as the central data store. The knowledge graph is the shared interface between ingestion (video or document), querying, exporting, and planning.
45	286
46		-1. Scan input directory for matching video files
47		-2. For each video: `process_single_video()` with error handling
48		-3. Merge knowledge graphs across all completed videos
49		-4. Generate batch summary with aggregated stats
50		-5. Write batch manifest
	287	+---
51	288
52	289	## Error handling
53	290
54		-- Individual video failures don't stop the batch
55		-- Failed videos are logged with error details in the manifest
56		-- Diagram analysis failures fall back to screengrabs
57		-- LLM extraction failures return empty results gracefully
	291	+Error handling follows consistent patterns across all pipelines:
	292	+
	293	+\| Scenario \| Behavior \|
	294	+\|----------\|----------\|
	295	+\| Video fails in batch \| Batch continues. Failed video recorded in manifest with error details. \|
	296	+\| Diagram analysis fails \| Falls back to screengrab (captioned screenshot). \|
	297	+\| LLM extraction fails \| Returns empty results gracefully. Key points and action items will be empty arrays. \|
	298	+\| Document processor not found \| Raises `ValueError` with list of supported extensions. \|
	299	+\| Source authentication fails \| Returns `False` from `authenticate()`. CLI prints error message. \|
	300	+\| Checkpoint file found \| Step is skipped entirely and results are loaded from disk. \|
	301	+\| Progress callback fails \| Warning logged. Pipeline continues without progress updates. \|
	302	+
	303	+---
	304	+
	305	+## Progress callback system
	306	+
	307	+The pipeline supports a `ProgressCallback` protocol for real-time progress tracking. This is used by the CLI's progress bars and can be implemented by external integrations (web UIs, CI systems, etc.).
	308	+
	309	+```python
	310	+from video_processor.models import ProgressCallback
	311	+
	312	+class MyCallback:
	313	+ def on_step_start(self, step: str, index: int, total: int) -> None:
	314	+ print(f"Starting step {index}/{total}: {step}")
	315	+
	316	+ def on_step_complete(self, step: str, index: int, total: int) -> None:
	317	+ print(f"Completed step {index}/{total}: {step}")
	318	+
	319	+ def on_progress(self, step: str, percent: float, message: str = "") -> None:
	320	+ print(f" {step}: {percent:.0%} {message}")
	321	+```
	322	+
	323	+Pass the callback to `process_single_video()`:
	324	+
	325	+```python
	326	+from video_processor.pipeline import process_single_video
	327	+
	328	+manifest = process_single_video(
	329	+ input_path="recording.mp4",
	330	+ output_dir="./output",
	331	+ progress_callback=MyCallback(),
	332	+)
	333	+```
	334	+
	335	+The callback methods are called within a try/except wrapper, so a failing callback never interrupts the pipeline. If a callback method raises an exception, a warning is logged and processing continues.
58	336

	--- docs/architecture/pipeline.md
	+++ docs/architecture/pipeline.md
	@@ -1,8 +1,14 @@
1	# Processing Pipeline




2
3	## Single video pipeline


4
5	```mermaid
6	sequenceDiagram
7	participant CLI
8	participant Pipeline
	@@ -9,49 +15,321 @@
9	participant FrameExtractor
10	participant AudioExtractor
11	participant Provider
12	participant DiagramAnalyzer
13	participant KnowledgeGraph

14
15	CLI->>Pipeline: process_single_video()


16	Pipeline->>FrameExtractor: extract_frames()
17	Note over FrameExtractor: Change detection + periodic capture (every 30s)



18	Pipeline->>Pipeline: filter_people_frames()
19	Note over Pipeline: OpenCV face detection removes webcam/people frames


20	Pipeline->>AudioExtractor: extract_audio()
21	Pipeline->>Provider: transcribe_audio()



22	Pipeline->>DiagramAnalyzer: process_frames()
23
24	loop Each frame
25	DiagramAnalyzer->>Provider: classify (vision)
26	alt High confidence diagram
27	DiagramAnalyzer->>Provider: full analysis

28	else Medium confidence
29	DiagramAnalyzer-->>Pipeline: screengrab fallback
30	end
31	end
32


33	Pipeline->>KnowledgeGraph: process_transcript()
34	Pipeline->>KnowledgeGraph: process_diagrams()



35	Pipeline->>Provider: extract key points
36	Pipeline->>Provider: extract action items
37	Pipeline->>Pipeline: generate reports
38	Pipeline->>Pipeline: export formats







39	Pipeline-->>CLI: VideoManifest
40	```















41
42	## Batch pipeline
43
44	The batch command wraps the single-video pipeline:





































































































































































































45
46	1. Scan input directory for matching video files
47	2. For each video: `process_single_video()` with error handling
48	3. Merge knowledge graphs across all completed videos
49	4. Generate batch summary with aggregated stats
50	5. Write batch manifest
51
52	## Error handling
53
54	- Individual video failures don't stop the batch
55	- Failed videos are logged with error details in the manifest
56	- Diagram analysis failures fall back to screengrabs
57	- LLM extraction failures return empty results gracefully









































58

	--- docs/architecture/pipeline.md
	+++ docs/architecture/pipeline.md
	@@ -1,8 +1,14 @@
1	# Processing Pipeline
2
3	PlanOpticon has four main pipelines: video analysis, document ingestion, source connector, and export. Each pipeline can operate independently, and they connect through the shared knowledge graph.
4
5	---
6
7	## Single video pipeline
8
9	The core video analysis pipeline processes a single video file through eight sequential steps with checkpoint/resume support.
10
11	```mermaid
12	sequenceDiagram
13	participant CLI
14	participant Pipeline
	@@ -9,49 +15,321 @@
15	participant FrameExtractor
16	participant AudioExtractor
17	participant Provider
18	participant DiagramAnalyzer
19	participant KnowledgeGraph
20	participant Exporter
21
22	CLI->>Pipeline: process_single_video()
23
24	Note over Pipeline: Step 1: Extract frames
25	Pipeline->>FrameExtractor: extract_frames()
26	Note over FrameExtractor: Change detection + periodic capture (every 30s)
27	FrameExtractor-->>Pipeline: frame_paths[]
28
29	Note over Pipeline: Step 2: Filter people frames
30	Pipeline->>Pipeline: filter_people_frames()
31	Note over Pipeline: OpenCV face detection removes webcam/people frames
32
33	Note over Pipeline: Step 3: Extract + transcribe audio
34	Pipeline->>AudioExtractor: extract_audio()
35	Pipeline->>Provider: transcribe_audio()
36	Note over Provider: Supports speaker hints via --speakers flag
37
38	Note over Pipeline: Step 4: Analyze visuals
39	Pipeline->>DiagramAnalyzer: process_frames()
40	loop Each frame (up to 10 standard / 20 comprehensive)

41	DiagramAnalyzer->>Provider: classify (vision)
42	alt High confidence diagram
43	DiagramAnalyzer->>Provider: full analysis
44	Note over Provider: Extract description, text, mermaid, chart data
45	else Medium confidence
46	DiagramAnalyzer-->>Pipeline: screengrab fallback
47	end
48	end
49
50	Note over Pipeline: Step 5: Build knowledge graph
51	Pipeline->>KnowledgeGraph: register_source()
52	Pipeline->>KnowledgeGraph: process_transcript()
53	Pipeline->>KnowledgeGraph: process_diagrams()
54	Note over KnowledgeGraph: Writes knowledge_graph.db (SQLite) + .json
55
56	Note over Pipeline: Step 6: Extract key points + action items
57	Pipeline->>Provider: extract key points
58	Pipeline->>Provider: extract action items
59
60	Note over Pipeline: Step 7: Generate report
61	Pipeline->>Pipeline: generate markdown report
62	Note over Pipeline: Includes mermaid diagrams, tables, cross-references
63
64	Note over Pipeline: Step 8: Export formats
65	Pipeline->>Exporter: export_all_formats()
66	Note over Exporter: HTML report, PDF, SVG/PNG renderings, chart reproductions
67
68	Pipeline-->>CLI: VideoManifest
69	```
70
71	### Pipeline steps in detail
72
73	\| Step \| Name \| Checkpointable \| Description \|
74	\|------\|------\|----------------\|-------------\|
75	\| 1 \| Extract frames \| Yes \| Change detection + periodic capture. Skipped if `frames/frame_*.jpg` exist on disk. \|
76	\| 2 \| Filter people frames \| No \| Inline with step 1. OpenCV face detection removes webcam frames. \|
77	\| 3 \| Extract + transcribe audio \| Yes \| Skipped if `transcript/transcript.json` exists. Speaker hints passed if `--speakers` provided. \|
78	\| 4 \| Analyze visuals \| Yes \| Skipped if `diagrams/` is populated. Evenly samples frames (not just first N). \|
79	\| 5 \| Build knowledge graph \| Yes \| Skipped if `results/knowledge_graph.db` exists. Registers source, processes transcript and diagrams. \|
80	\| 6 \| Extract key points + actions \| Yes \| Skipped if `results/key_points.json` and `results/action_items.json` exist. \|
81	\| 7 \| Generate report \| Yes \| Skipped if `results/analysis.md` exists. \|
82	\| 8 \| Export formats \| No \| Always runs. Renders mermaid to SVG/PNG, reproduces charts, generates HTML/PDF. \|
83
84	---
85
86	## Batch pipeline
87
88	The batch pipeline wraps the single-video pipeline and adds cross-video knowledge graph merging.
89
90	```mermaid
91	flowchart TD
92	A[Scan input directory] --> B[Match video files by pattern]
93	B --> C{For each video}
94	C --> D[process_single_video]
95	D --> E{Success?}
96	E -->\|Yes\| F[Collect manifest + KG]
97	E -->\|No\| G[Log error, continue]
98	F --> H[Next video]
99	G --> H
100	H --> C
101	C -->\|All done\| I[Merge knowledge graphs]
102	I --> J[Fuzzy matching + conflict resolution]
103	J --> K[Generate batch summary]
104	K --> L[Write batch manifest]
105	L --> M[batch_manifest.json + batch_summary.md + merged KG]
106	```
107
108	### Knowledge graph merge strategy
109
110	During batch merging, `KnowledgeGraph.merge()` applies:
111
112	1. Case-insensitive exact matching for entity names
113	2. Fuzzy matching via `SequenceMatcher` (threshold >= 0.85) for near-duplicates
114	3. Type conflict resolution using a specificity ranking (e.g., `technology` > `concept`)
115	4. Description union across all sources
116	5. Relationship deduplication by (source, target, type) tuple
117
118	---
119
120	## Document ingestion pipeline
121
122	The document ingestion pipeline processes files (Markdown, plaintext, PDF) into knowledge graphs without video analysis.
123
124	```mermaid
125	flowchart TD
126	A[Input: file or directory] --> B{File or directory?}
127	B -->\|File\| C[get_processor by extension]
128	B -->\|Directory\| D[Glob for supported extensions]
129	D --> E{Recursive?}
130	E -->\|Yes\| F[rglob all files]
131	E -->\|No\| G[glob top-level only]
132	F --> H[For each file]
133	G --> H
134	H --> C
135	C --> I[DocumentProcessor.process]
136	I --> J[DocumentChunk list]
137	J --> K[Register source in KG]
138	K --> L[Add chunks as content]
139	L --> M[KG extracts entities + relationships]
140	M --> N[knowledge_graph.db]
141	```
142
143	### Supported document types
144
145	\| Extension \| Processor \| Notes \|
146	\|-----------\|-----------\|-------\|
147	\| `.md` \| `MarkdownProcessor` \| Splits by headings into sections \|
148	\| `.txt` \| `PlaintextProcessor` \| Splits into fixed-size chunks \|
149	\| `.pdf` \| `PdfProcessor` \| Requires `pymupdf` or `pdfplumber`. Falls back gracefully between libraries. \|
150
151	### Adding documents to an existing graph
152
153	The `--db-path` flag lets you ingest documents into an existing knowledge graph:
154
155	```bash
156	planopticon ingest spec.md --db-path existing.db
157	planopticon ingest ./docs/ -o ./output --recursive
158	```
159
160	---
161
162	## Source connector pipeline
163
164	Source connectors fetch content from cloud services, note-taking apps, and web sources. Each source implements the `BaseSource` ABC with three methods: `authenticate()`, `list_videos()`, and `download()`.
165
166	```mermaid
167	flowchart TD
168	A[Source command] --> B[Authenticate with provider]
169	B --> C{Auth success?}
170	C -->\|No\| D[Error: check credentials]
171	C -->\|Yes\| E[List files in folder]
172	E --> F[Filter by pattern / type]
173	F --> G[Download to local path]
174	G --> H{Analyze or ingest?}
175	H -->\|Video\| I[process_single_video / batch]
176	H -->\|Document\| J[ingest_file / ingest_directory]
177	I --> K[Knowledge graph]
178	J --> K
179	```
180
181	### Available sources
182
183	PlanOpticon includes connectors for:
184
185	\| Category \| Sources \|
186	\|----------\|---------\|
187	\| Cloud storage \| Google Drive, S3, Dropbox \|
188	\| Meeting recordings \| Zoom, Google Meet, Microsoft Teams \|
189	\| Productivity suites \| Google Workspace (Docs/Sheets/Slides), Microsoft 365 (SharePoint/OneDrive/OneNote) \|
190	\| Note-taking apps \| Obsidian, Logseq, Apple Notes, Google Keep, Notion \|
191	\| Web sources \| YouTube, Web (URL), RSS, Podcasts \|
192	\| Developer platforms \| GitHub, arXiv \|
193	\| Social media \| Reddit, Twitter/X, Hacker News \|
194
195	Each source authenticates via environment variables (API keys, OAuth tokens) specific to the provider.
196
197	---
198
199	## Planning agent pipeline
200
201	The planning agent consumes a knowledge graph and uses registered skills to generate planning artifacts.
202
203	```mermaid
204	flowchart TD
205	A[Knowledge graph] --> B[Load into AgentContext]
206	B --> C[GraphQueryEngine]
207	C --> D[Taxonomy classification]
208	D --> E[Agent orchestrator]
209	E --> F{Select skill}
210	F --> G[ProjectPlan skill]
211	F --> H[PRD skill]
212	F --> I[Roadmap skill]
213	F --> J[TaskBreakdown skill]
214	F --> K[DocGenerator skill]
215	F --> L[WikiGenerator skill]
216	F --> M[NotesExport skill]
217	F --> N[ArtifactExport skill]
218	F --> O[GitHubIntegration skill]
219	F --> P[RequirementsChat skill]
220	G --> Q[Artifact output]
221	H --> Q
222	I --> Q
223	J --> Q
224	K --> Q
225	L --> Q
226	M --> Q
227	N --> Q
228	O --> Q
229	P --> Q
230	Q --> R[Write to disk / push to service]
231	```
232
233	### Skill execution flow
234
235	1. The `AgentContext` is populated with the knowledge graph, query engine, provider manager, and any planning entities from taxonomy classification
236	2. Each `Skill` checks `can_execute()` against the context (requires at minimum a knowledge graph and provider manager)
237	3. The skill's `execute()` method generates an `Artifact` with a name, content, type, and format
238	4. Artifacts are collected and can be exported to disk or pushed to external services (GitHub issues, wiki pages, etc.)
239
240	---
241
242	## Export pipeline
243
244	The export pipeline converts knowledge graphs and analysis artifacts into various output formats.
245
246	```mermaid
247	flowchart TD
248	A[knowledge_graph.db] --> B{Export command}
249	B --> C[export markdown]
250	B --> D[export obsidian]
251	B --> E[export notion]
252	B --> F[export exchange]
253	B --> G[wiki generate]
254	B --> H[kg convert]
255	C --> I[7 document types + entity briefs + CSV]
256	D --> J[Obsidian vault with frontmatter + wiki-links]
257	E --> K[Notion-compatible markdown + CSV database]
258	F --> L[PlanOpticonExchange JSON payload]
259	G --> M[GitHub wiki pages + sidebar + home]
260	H --> N[Convert between .db / .json / .graphml / .csv]
261	```
262
263	All export commands accept a `knowledge_graph.db` (or `.json`) path as input. No API key is required for template-based exports (markdown, obsidian, notion, wiki, exchange, convert). Only the planning agent skills that generate new content require a provider.
264
265	---
266
267	## How pipelines connect
268
269	```mermaid
270	flowchart LR
271	V[Video files] --> VP[Video Pipeline]
272	D[Documents] --> DI[Document Ingestion]
273	S[Cloud Sources] --> SC[Source Connectors]
274	SC --> V
275	SC --> D
276	VP --> KG[(knowledge_graph.db)]
277	DI --> KG
278	KG --> QE[Query Engine]
279	KG --> EP[Export Pipeline]
280	KG --> PA[Planning Agent]
281	PA --> AR[Artifacts]
282	AR --> EP
283	```
284
285	All pipelines converge on the knowledge graph as the central data store. The knowledge graph is the shared interface between ingestion (video or document), querying, exporting, and planning.
286
287	---




288
289	## Error handling
290
291	Error handling follows consistent patterns across all pipelines:
292
293	\| Scenario \| Behavior \|
294	\|----------\|----------\|
295	\| Video fails in batch \| Batch continues. Failed video recorded in manifest with error details. \|
296	\| Diagram analysis fails \| Falls back to screengrab (captioned screenshot). \|
297	\| LLM extraction fails \| Returns empty results gracefully. Key points and action items will be empty arrays. \|
298	\| Document processor not found \| Raises `ValueError` with list of supported extensions. \|
299	\| Source authentication fails \| Returns `False` from `authenticate()`. CLI prints error message. \|
300	\| Checkpoint file found \| Step is skipped entirely and results are loaded from disk. \|
301	\| Progress callback fails \| Warning logged. Pipeline continues without progress updates. \|
302
303	---
304
305	## Progress callback system
306
307	The pipeline supports a `ProgressCallback` protocol for real-time progress tracking. This is used by the CLI's progress bars and can be implemented by external integrations (web UIs, CI systems, etc.).
308
309	```python
310	from video_processor.models import ProgressCallback
311
312	class MyCallback:
313	def on_step_start(self, step: str, index: int, total: int) -> None:
314	print(f"Starting step {index}/{total}: {step}")
315
316	def on_step_complete(self, step: str, index: int, total: int) -> None:
317	print(f"Completed step {index}/{total}: {step}")
318
319	def on_progress(self, step: str, percent: float, message: str = "") -> None:
320	print(f" {step}: {percent:.0%} {message}")
321	```
322
323	Pass the callback to `process_single_video()`:
324
325	```python
326	from video_processor.pipeline import process_single_video
327
328	manifest = process_single_video(
329	input_path="recording.mp4",
330	output_dir="./output",
331	progress_callback=MyCallback(),
332	)
333	```
334
335	The callback methods are called within a try/except wrapper, so a failing callback never interrupts the pipeline. If a callback method raises an exception, a warning is logged and processing continues.
336

M docs/contributing.md

+444 -13

		--- docs/contributing.md
		+++ docs/contributing.md
		@@ -10,54 +10,485 @@
10	10	pip install -e ".[dev]"
11	11	```
12	12
13	13	## Running tests
14	14
	15	+PlanOpticon has 822+ tests covering providers, pipeline stages, document processors, knowledge graph operations, exporters, skills, and CLI commands.
	16	+
15	17	```bash
16	18	# Run all tests
17	19	pytest tests/ -v
18	20
19	21	# Run with coverage
20	22	pytest tests/ --cov=video_processor --cov-report=html
21	23
22	24	# Run a specific test file
23	25	pytest tests/test_models.py -v
	26	+
	27	+# Run tests matching a keyword
	28	+pytest tests/ -k "test_knowledge_graph" -v
	29	+
	30	+# Run only fast tests (skip slow integration tests)
	31	+pytest tests/ -m "not slow" -v
	32	+```
	33	+
	34	+### Test conventions
	35	+
	36	+- All tests live in the `tests/` directory, mirroring the `video_processor/` package structure
	37	+- Test files are named `test_<module>.py`
	38	+- Use `pytest` as the test runner -- do not use `unittest.TestCase` unless necessary for specific setup/teardown patterns
	39	+- Mock external API calls. Never make real API calls in tests. Use `unittest.mock.patch` or `pytest-mock` fixtures to mock provider responses.
	40	+- Use `tmp_path` (pytest fixture) for any tests that write files to disk
	41	+- Fixtures shared across test files go in `conftest.py`
	42	+- For testing CLI commands, use `click.testing.CliRunner`
	43	+- For testing provider implementations, mock at the HTTP client level (e.g., patch `requests.post` or the provider's SDK client)
	44	+
	45	+### Mocking patterns
	46	+
	47	+```python
	48	+# Mocking a provider's chat method
	49	+from unittest.mock import MagicMock, patch
	50	+
	51	+def test_key_point_extraction():
	52	+ pm = MagicMock()
	53	+ pm.chat.return_value = '["Point 1", "Point 2"]'
	54	+ result = extract_key_points(pm, "transcript text")
	55	+ assert len(result) == 2
	56	+
	57	+# Mocking an external API at the HTTP level
	58	+@patch("requests.post")
	59	+def test_provider_chat(mock_post):
	60	+ mock_post.return_value.json.return_value = {
	61	+ "choices": [{"message": {"content": "response"}}]
	62	+ }
	63	+ provider = OpenAIProvider(api_key="test")
	64	+ result = provider.chat([{"role": "user", "content": "hello"}])
	65	+ assert result == "response"
24	66	```
25	67
26	68	## Code style
27	69
28	70	We use:
29	71
30		-- Ruff for linting
31		-- Black for formatting (100 char line length)
32		-- isort for import sorting
	72	+- Ruff for both linting and formatting (100 char line length)
33	73	- mypy for type checking
	74	+
	75	+Ruff handles all linting (error, warning, pyflakes, and import sorting rules) and formatting in a single tool. There is no need to run Black or isort separately.
34	76
35	77	```bash
	78	+# Lint
36	79	ruff check video_processor/
37		-black video_processor/
38		-isort video_processor/
	80	+
	81	+# Format
	82	+ruff format video_processor/
	83	+
	84	+# Auto-fix lint issues
	85	+ruff check video_processor/ --fix
	86	+
	87	+# Type check
39	88	mypy video_processor/ --ignore-missing-imports
40	89	```
41	90
	91	+### Ruff configuration
	92	+
	93	+The project's `pyproject.toml` configures ruff as follows:
	94	+
	95	+```toml
	96	+[tool.ruff]
	97	+line-length = 100
	98	+target-version = "py310"
	99	+
	100	+[tool.ruff.lint]
	101	+select = ["E", "F", "W", "I"]
	102	+```
	103	+
	104	+The `I` rule set covers import sorting (equivalent to isort), so imports are automatically organized by ruff.
	105	+
42	106	## Project structure
43	107
44		-See [Architecture Overview](architecture/overview.md) for the module structure.
	108	+```
	109	+PlanOpticon/
	110	+├── video_processor/
	111	+│ ├── cli/ # Click CLI commands
	112	+│ │ └── commands.py
	113	+│ ├── providers/ # LLM/API provider implementations
	114	+│ │ ├── base.py # BaseProvider, ProviderRegistry
	115	+│ │ ├── manager.py # ProviderManager
	116	+│ │ ├── discovery.py # Auto-discovery of available providers
	117	+│ │ ├── openai_provider.py
	118	+│ │ ├── anthropic_provider.py
	119	+│ │ ├── gemini_provider.py
	120	+│ │ └── ... # 15+ provider implementations
	121	+│ ├── sources/ # Cloud and web source connectors
	122	+│ │ ├── base.py # BaseSource, SourceFile
	123	+│ │ ├── google_drive.py
	124	+│ │ ├── zoom_source.py
	125	+│ │ └── ... # 20+ source implementations
	126	+│ ├── processors/ # Document processors
	127	+│ │ ├── base.py # DocumentProcessor, registry
	128	+│ │ ├── ingest.py # File/directory ingestion
	129	+│ │ ├── markdown_processor.py
	130	+│ │ ├── pdf_processor.py
	131	+│ │ └── __init__.py # Auto-registration of built-in processors
	132	+│ ├── integrators/ # Knowledge graph and analysis
	133	+│ │ ├── knowledge_graph.py # KnowledgeGraph class
	134	+│ │ ├── graph_store.py # SQLite graph storage
	135	+│ │ ├── graph_query.py # GraphQueryEngine
	136	+│ │ ├── graph_discovery.py # Auto-find knowledge_graph.db
	137	+│ │ └── taxonomy.py # Planning taxonomy classifier
	138	+│ ├── agent/ # Planning agent
	139	+│ │ ├── orchestrator.py # Agent orchestration
	140	+│ │ └── skills/ # Skill implementations
	141	+│ │ ├── base.py # Skill ABC, registry, Artifact
	142	+│ │ ├── project_plan.py
	143	+│ │ ├── prd.py
	144	+│ │ ├── roadmap.py
	145	+│ │ ├── task_breakdown.py
	146	+│ │ ├── doc_generator.py
	147	+│ │ ├── wiki_generator.py
	148	+│ │ ├── notes_export.py
	149	+│ │ ├── artifact_export.py
	150	+│ │ ├── github_integration.py
	151	+│ │ ├── requirements_chat.py
	152	+│ │ ├── cli_adapter.py
	153	+│ │ └── __init__.py # Auto-registration of skills
	154	+│ ├── exporters/ # Output format exporters
	155	+│ │ ├── __init__.py
	156	+│ │ └── markdown.py # Template-based markdown generation
	157	+│ ├── utils/ # Shared utilities
	158	+│ │ ├── export.py # Multi-format export orchestration
	159	+│ │ ├── rendering.py # Mermaid/chart rendering
	160	+│ │ ├── prompt_templates.py
	161	+│ │ ├── callbacks.py # Progress callback helpers
	162	+│ │ └── ...
	163	+│ ├── exchange.py # PlanOpticonExchange format
	164	+│ ├── pipeline.py # Main video processing pipeline
	165	+│ ├── models.py # Pydantic data models
	166	+│ └── output_structure.py # Output directory helpers
	167	+├── tests/ # 822+ tests
	168	+├── knowledge-base/ # Local-first graph tools
	169	+│ ├── viewer.html # Self-contained D3.js graph viewer
	170	+│ └── query.py # Python query script (NetworkX)
	171	+├── docs/ # MkDocs documentation
	172	+└── pyproject.toml # Project configuration
	173	+```
	174	+
	175	+See [Architecture Overview](architecture/overview.md) for a more detailed breakdown of module responsibilities.
45	176
46	177	## Adding a new provider
47	178
	179	+Providers self-register via `ProviderRegistry.register()` at module level. When the provider module is imported, it registers itself automatically.
	180	+
48	181	1. Create `video_processor/providers/your_provider.py`
49	182	2. Extend `BaseProvider` from `video_processor/providers/base.py`
50		-3. Implement `chat()`, `analyze_image()`, `transcribe_audio()`, `list_models()`
51		-4. Register in `video_processor/providers/discovery.py`
52		-5. Add tests in `tests/test_providers.py`
	183	+3. Implement the four required methods: `chat()`, `analyze_image()`, `transcribe_audio()`, `list_models()`
	184	+4. Call `ProviderRegistry.register()` at module level
	185	+5. Add the import to `video_processor/providers/manager.py` in the lazy-import block
	186	+6. Add tests in `tests/test_providers.py`
	187	+
	188	+### Example provider skeleton
	189	+
	190	+```python
	191	+"""Your provider implementation."""
	192	+
	193	+from video_processor.providers.base import BaseProvider, ModelInfo, ProviderRegistry
	194	+
	195	+
	196	+class YourProvider(BaseProvider):
	197	+ provider_name = "yourprovider"
	198	+
	199	+ def __init__(self, api_key: str \| None = None):
	200	+ import os
	201	+ self.api_key = api_key or os.environ.get("YOUR_API_KEY", "")
	202	+
	203	+ def chat(self, messages, max_tokens=4096, temperature=0.7, model=None):
	204	+ # Implement chat completion
	205	+ ...
	206	+
	207	+ def analyze_image(self, image_bytes, prompt, max_tokens=4096, model=None):
	208	+ # Implement image analysis
	209	+ ...
	210	+
	211	+ def transcribe_audio(self, audio_path, language=None, model=None):
	212	+ # Implement audio transcription (or raise NotImplementedError)
	213	+ ...
	214	+
	215	+ def list_models(self):
	216	+ return [ModelInfo(id="your-model", provider="yourprovider", capabilities=["chat"])]
	217	+
	218	+
	219	+# Self-registration at import time
	220	+ProviderRegistry.register(
	221	+ "yourprovider",
	222	+ YourProvider,
	223	+ env_var="YOUR_API_KEY",
	224	+ model_prefixes=["your-"],
	225	+ default_models={"chat": "your-model"},
	226	+)
	227	+```
	228	+
	229	+### OpenAI-compatible providers
	230	+
	231	+For providers that use the OpenAI API format, extend `OpenAICompatibleProvider` instead of `BaseProvider`. This provides default implementations of `chat()`, `analyze_image()`, and `list_models()` -- you only need to configure the base URL and model mappings.
	232	+
	233	+```python
	234	+from video_processor.providers.base import OpenAICompatibleProvider, ProviderRegistry
	235	+
	236	+class YourProvider(OpenAICompatibleProvider):
	237	+ provider_name = "yourprovider"
	238	+ base_url = "https://api.yourprovider.com/v1"
	239	+ env_var = "YOUR_API_KEY"
	240	+
	241	+ProviderRegistry.register("yourprovider", YourProvider, env_var="YOUR_API_KEY")
	242	+```
53	243
54	244	## Adding a new cloud source
55	245
	246	+Source connectors implement the `BaseSource` ABC from `video_processor/sources/base.py`. Authentication is handled per-source, typically via environment variables.
	247	+
56	248	1. Create `video_processor/sources/your_source.py`
57		-2. Implement auth flow and file listing/downloading
58		-3. Add CLI integration in `video_processor/cli/commands.py`
59		-4. Add tests and docs
	249	+2. Extend `BaseSource`
	250	+3. Implement `authenticate()`, `list_videos()`, and `download()`
	251	+4. Add the class to the lazy-import map in `video_processor/sources/__init__.py`
	252	+5. Add CLI commands in `video_processor/cli/commands.py` if needed
	253	+6. Add tests and documentation
	254	+
	255	+### Example source skeleton
	256	+
	257	+```python
	258	+"""Your source integration."""
	259	+
	260	+import os
	261	+import logging
	262	+from pathlib import Path
	263	+from typing import List, Optional
	264	+
	265	+from video_processor.sources.base import BaseSource, SourceFile
	266	+
	267	+logger = logging.getLogger(__name__)
	268	+
	269	+
	270	+class YourSource(BaseSource):
	271	+ def __init__(self, api_key: Optional[str] = None):
	272	+ self.api_key = api_key or os.environ.get("YOUR_SOURCE_KEY", "")
	273	+
	274	+ def authenticate(self) -> bool:
	275	+ """Validate credentials. Return True on success."""
	276	+ if not self.api_key:
	277	+ logger.error("API key not set. Set YOUR_SOURCE_KEY env var.")
	278	+ return False
	279	+ # Make a test API call to verify credentials
	280	+ ...
	281	+ return True
	282	+
	283	+ def list_videos(
	284	+ self,
	285	+ folder_id: Optional[str] = None,
	286	+ folder_path: Optional[str] = None,
	287	+ patterns: Optional[List[str]] = None,
	288	+ ) -> List[SourceFile]:
	289	+ """List available video files."""
	290	+ ...
	291	+
	292	+ def download(self, file: SourceFile, destination: Path) -> Path:
	293	+ """Download a single file. Return the local path."""
	294	+ destination.parent.mkdir(parents=True, exist_ok=True)
	295	+ # Download file content to destination
	296	+ ...
	297	+ return destination
	298	+```
	299	+
	300	+### Registering in `__init__.py`
	301	+
	302	+Add your source to the `__all__` list and the `_lazy_map` dictionary in `video_processor/sources/__init__.py`:
	303	+
	304	+```python
	305	+__all__ = [
	306	+ ...
	307	+ "YourSource",
	308	+]
	309	+
	310	+_lazy_map = {
	311	+ ...
	312	+ "YourSource": "video_processor.sources.your_source",
	313	+}
	314	+```
	315	+
	316	+## Adding a new skill
	317	+
	318	+Agent skills extend the `Skill` ABC from `video_processor/agent/skills/base.py` and self-register via `register_skill()`.
	319	+
	320	+1. Create `video_processor/agent/skills/your_skill.py`
	321	+2. Extend `Skill` and set `name` and `description` class attributes
	322	+3. Implement `execute()` to return an `Artifact`
	323	+4. Optionally override `can_execute()` for custom precondition checks
	324	+5. Call `register_skill()` at module level
	325	+6. Add the import to `video_processor/agent/skills/__init__.py`
	326	+7. Add tests
	327	+
	328	+### Example skill skeleton
	329	+
	330	+```python
	331	+"""Your custom skill."""
	332	+
	333	+from video_processor.agent.skills.base import AgentContext, Artifact, Skill, register_skill
	334	+
	335	+
	336	+class YourSkill(Skill):
	337	+ name = "your_skill"
	338	+ description = "Generates a custom artifact from the knowledge graph."
	339	+
	340	+ def execute(self, context: AgentContext, **kwargs) -> Artifact:
	341	+ """Generate the artifact."""
	342	+ kg_data = context.knowledge_graph.to_dict()
	343	+ # Build content from knowledge graph data
	344	+ content = f"# Your Artifact\n\n{len(kg_data.get('entities', []))} entities found."
	345	+ return Artifact(
	346	+ name="your_artifact",
	347	+ content=content,
	348	+ artifact_type="document",
	349	+ format="markdown",
	350	+ )
	351	+
	352	+ def can_execute(self, context: AgentContext) -> bool:
	353	+ """Check prerequisites (default requires KG + provider)."""
	354	+ return context.knowledge_graph is not None
	355	+
	356	+
	357	+# Self-registration at import time
	358	+register_skill(YourSkill())
	359	+```
	360	+
	361	+### Registering in `__init__.py`
	362	+
	363	+Add the import to `video_processor/agent/skills/__init__.py` so the skill is loaded (and self-registered) when the skills package is imported:
	364	+
	365	+```python
	366	+from video_processor.agent.skills import (
	367	+ ...
	368	+ your_skill, # noqa: F401
	369	+)
	370	+```
	371	+
	372	+## Adding a new document processor
	373	+
	374	+Document processors extend the `DocumentProcessor` ABC from `video_processor/processors/base.py` and are registered via `register_processor()`.
	375	+
	376	+1. Create `video_processor/processors/your_processor.py`
	377	+2. Extend `DocumentProcessor`
	378	+3. Set `supported_extensions` class attribute
	379	+4. Implement `process()` (returns `List[DocumentChunk]`) and `can_process()`
	380	+5. Call `register_processor()` at module level
	381	+6. Add the import to `video_processor/processors/__init__.py`
	382	+7. Add tests
	383	+
	384	+### Example processor skeleton
	385	+
	386	+```python
	387	+"""Your document processor."""
	388	+
	389	+from pathlib import Path
	390	+from typing import List
	391	+
	392	+from video_processor.processors.base import (
	393	+ DocumentChunk,
	394	+ DocumentProcessor,
	395	+ register_processor,
	396	+)
	397	+
	398	+
	399	+class YourProcessor(DocumentProcessor):
	400	+ supported_extensions = [".xyz", ".abc"]
	401	+
	402	+ def can_process(self, path: Path) -> bool:
	403	+ return path.suffix.lower() in self.supported_extensions
	404	+
	405	+ def process(self, path: Path) -> List[DocumentChunk]:
	406	+ text = path.read_text()
	407	+ # Split into chunks as appropriate for your format
	408	+ return [
	409	+ DocumentChunk(
	410	+ text=text,
	411	+ source_file=str(path),
	412	+ chunk_index=0,
	413	+ metadata={"format": "xyz"},
	414	+ )
	415	+ ]
	416	+
	417	+
	418	+# Self-registration at import time
	419	+register_processor([".xyz", ".abc"], YourProcessor)
	420	+```
	421	+
	422	+### Registering in `__init__.py`
	423	+
	424	+Add the import to `video_processor/processors/__init__.py`:
	425	+
	426	+```python
	427	+from video_processor.processors import (
	428	+ markdown_processor, # noqa: F401, E402
	429	+ pdf_processor, # noqa: F401, E402
	430	+ your_processor, # noqa: F401, E402
	431	+)
	432	+```
	433	+
	434	+## Adding a new exporter
	435	+
	436	+Exporters live in `video_processor/exporters/` and are typically called from CLI commands. There is no strict ABC for exporters -- they are plain functions that accept knowledge graph data and an output directory.
	437	+
	438	+1. Create `video_processor/exporters/your_exporter.py`
	439	+2. Implement one or more export functions that accept KG data (as a dict) and an output path
	440	+3. Add CLI integration in `video_processor/cli/commands.py` under the `export` group
	441	+4. Add tests
	442	+
	443	+### Example exporter skeleton
	444	+
	445	+```python
	446	+"""Your exporter."""
	447	+
	448	+import json
	449	+from pathlib import Path
	450	+from typing import List
	451	+
	452	+
	453	+def export_your_format(kg_data: dict, output_dir: Path) -> List[Path]:
	454	+ """Export knowledge graph data in your format.
	455	+
	456	+ Args:
	457	+ kg_data: Knowledge graph as a dict (from KnowledgeGraph.to_dict()).
	458	+ output_dir: Directory to write output files.
	459	+
	460	+ Returns:
	461	+ List of created file paths.
	462	+ """
	463	+ output_dir.mkdir(parents=True, exist_ok=True)
	464	+ created = []
	465	+
	466	+ output_file = output_dir / "export.xyz"
	467	+ output_file.write_text(json.dumps(kg_data, indent=2))
	468	+ created.append(output_file)
	469	+
	470	+ return created
	471	+```
	472	+
	473	+### Adding the CLI command
	474	+
	475	+Add a subcommand under the `export` group in `video_processor/cli/commands.py`:
	476	+
	477	+```python
	478	+@export.command("your-format")
	479	+@click.argument("db_path", type=click.Path(exists=True))
	480	+@click.option("-o", "--output", type=click.Path(), default=None)
	481	+def export_your_format_cmd(db_path, output):
	482	+ """Export knowledge graph in your format."""
	483	+ from video_processor.exporters.your_exporter import export_your_format
	484	+ from video_processor.integrators.knowledge_graph import KnowledgeGraph
	485	+
	486	+ kg = KnowledgeGraph(db_path=Path(db_path))
	487	+ out_dir = Path(output) if output else Path.cwd() / "your-export"
	488	+ created = export_your_format(kg.to_dict(), out_dir)
	489	+ click.echo(f"Exported {len(created)} files to {out_dir}/")
	490	+```
60	491
61	492	## License
62	493
63		-MIT License — Copyright (c) 2025 CONFLICT LLC. All rights reserved.
	494	+MIT License -- Copyright (c) 2026 CONFLICT LLC. All rights reserved.
64	495
65	496	ADDED docs/faq.md

	--- docs/contributing.md
	+++ docs/contributing.md
	@@ -10,54 +10,485 @@
10	pip install -e ".[dev]"
11	```
12
13	## Running tests
14


15	```bash
16	# Run all tests
17	pytest tests/ -v
18
19	# Run with coverage
20	pytest tests/ --cov=video_processor --cov-report=html
21
22	# Run a specific test file
23	pytest tests/test_models.py -v








































24	```
25
26	## Code style
27
28	We use:
29
30	- Ruff for linting
31	- Black for formatting (100 char line length)
32	- isort for import sorting
33	- mypy for type checking


34
35	```bash

36	ruff check video_processor/
37	black video_processor/
38	isort video_processor/






39	mypy video_processor/ --ignore-missing-imports
40	```
41















42	## Project structure
43
44	See [Architecture Overview](architecture/overview.md) for the module structure.



































































45
46	## Adding a new provider
47


48	1. Create `video_processor/providers/your_provider.py`
49	2. Extend `BaseProvider` from `video_processor/providers/base.py`
50	3. Implement `chat()`, `analyze_image()`, `transcribe_audio()`, `list_models()`
51	4. Register in `video_processor/providers/discovery.py`
52	5. Add tests in `tests/test_providers.py`

























































53
54	## Adding a new cloud source
55


56	1. Create `video_processor/sources/your_source.py`
57	2. Implement auth flow and file listing/downloading
58	3. Add CLI integration in `video_processor/cli/commands.py`
59	4. Add tests and docs















































































































































































































































60
61	## License
62
63	MIT License — Copyright (c) 2025 CONFLICT LLC. All rights reserved.
64
65	DDED docs/faq.md

	--- docs/contributing.md
	+++ docs/contributing.md
	@@ -10,54 +10,485 @@
10	pip install -e ".[dev]"
11	```
12
13	## Running tests
14
15	PlanOpticon has 822+ tests covering providers, pipeline stages, document processors, knowledge graph operations, exporters, skills, and CLI commands.
16
17	```bash
18	# Run all tests
19	pytest tests/ -v
20
21	# Run with coverage
22	pytest tests/ --cov=video_processor --cov-report=html
23
24	# Run a specific test file
25	pytest tests/test_models.py -v
26
27	# Run tests matching a keyword
28	pytest tests/ -k "test_knowledge_graph" -v
29
30	# Run only fast tests (skip slow integration tests)
31	pytest tests/ -m "not slow" -v
32	```
33
34	### Test conventions
35
36	- All tests live in the `tests/` directory, mirroring the `video_processor/` package structure
37	- Test files are named `test_<module>.py`
38	- Use `pytest` as the test runner -- do not use `unittest.TestCase` unless necessary for specific setup/teardown patterns
39	- Mock external API calls. Never make real API calls in tests. Use `unittest.mock.patch` or `pytest-mock` fixtures to mock provider responses.
40	- Use `tmp_path` (pytest fixture) for any tests that write files to disk
41	- Fixtures shared across test files go in `conftest.py`
42	- For testing CLI commands, use `click.testing.CliRunner`
43	- For testing provider implementations, mock at the HTTP client level (e.g., patch `requests.post` or the provider's SDK client)
44
45	### Mocking patterns
46
47	```python
48	# Mocking a provider's chat method
49	from unittest.mock import MagicMock, patch
50
51	def test_key_point_extraction():
52	pm = MagicMock()
53	pm.chat.return_value = '["Point 1", "Point 2"]'
54	result = extract_key_points(pm, "transcript text")
55	assert len(result) == 2
56
57	# Mocking an external API at the HTTP level
58	@patch("requests.post")
59	def test_provider_chat(mock_post):
60	mock_post.return_value.json.return_value = {
61	"choices": [{"message": {"content": "response"}}]
62	}
63	provider = OpenAIProvider(api_key="test")
64	result = provider.chat([{"role": "user", "content": "hello"}])
65	assert result == "response"
66	```
67
68	## Code style
69
70	We use:
71
72	- Ruff for both linting and formatting (100 char line length)


73	- mypy for type checking
74
75	Ruff handles all linting (error, warning, pyflakes, and import sorting rules) and formatting in a single tool. There is no need to run Black or isort separately.
76
77	```bash
78	# Lint
79	ruff check video_processor/
80
81	# Format
82	ruff format video_processor/
83
84	# Auto-fix lint issues
85	ruff check video_processor/ --fix
86
87	# Type check
88	mypy video_processor/ --ignore-missing-imports
89	```
90
91	### Ruff configuration
92
93	The project's `pyproject.toml` configures ruff as follows:
94
95	```toml
96	[tool.ruff]
97	line-length = 100
98	target-version = "py310"
99
100	[tool.ruff.lint]
101	select = ["E", "F", "W", "I"]
102	```
103
104	The `I` rule set covers import sorting (equivalent to isort), so imports are automatically organized by ruff.
105
106	## Project structure
107
108	```
109	PlanOpticon/
110	├── video_processor/
111	│ ├── cli/ # Click CLI commands
112	│ │ └── commands.py
113	│ ├── providers/ # LLM/API provider implementations
114	│ │ ├── base.py # BaseProvider, ProviderRegistry
115	│ │ ├── manager.py # ProviderManager
116	│ │ ├── discovery.py # Auto-discovery of available providers
117	│ │ ├── openai_provider.py
118	│ │ ├── anthropic_provider.py
119	│ │ ├── gemini_provider.py
120	│ │ └── ... # 15+ provider implementations
121	│ ├── sources/ # Cloud and web source connectors
122	│ │ ├── base.py # BaseSource, SourceFile
123	│ │ ├── google_drive.py
124	│ │ ├── zoom_source.py
125	│ │ └── ... # 20+ source implementations
126	│ ├── processors/ # Document processors
127	│ │ ├── base.py # DocumentProcessor, registry
128	│ │ ├── ingest.py # File/directory ingestion
129	│ │ ├── markdown_processor.py
130	│ │ ├── pdf_processor.py
131	│ │ └── __init__.py # Auto-registration of built-in processors
132	│ ├── integrators/ # Knowledge graph and analysis
133	│ │ ├── knowledge_graph.py # KnowledgeGraph class
134	│ │ ├── graph_store.py # SQLite graph storage
135	│ │ ├── graph_query.py # GraphQueryEngine
136	│ │ ├── graph_discovery.py # Auto-find knowledge_graph.db
137	│ │ └── taxonomy.py # Planning taxonomy classifier
138	│ ├── agent/ # Planning agent
139	│ │ ├── orchestrator.py # Agent orchestration
140	│ │ └── skills/ # Skill implementations
141	│ │ ├── base.py # Skill ABC, registry, Artifact
142	│ │ ├── project_plan.py
143	│ │ ├── prd.py
144	│ │ ├── roadmap.py
145	│ │ ├── task_breakdown.py
146	│ │ ├── doc_generator.py
147	│ │ ├── wiki_generator.py
148	│ │ ├── notes_export.py
149	│ │ ├── artifact_export.py
150	│ │ ├── github_integration.py
151	│ │ ├── requirements_chat.py
152	│ │ ├── cli_adapter.py
153	│ │ └── __init__.py # Auto-registration of skills
154	│ ├── exporters/ # Output format exporters
155	│ │ ├── __init__.py
156	│ │ └── markdown.py # Template-based markdown generation
157	│ ├── utils/ # Shared utilities
158	│ │ ├── export.py # Multi-format export orchestration
159	│ │ ├── rendering.py # Mermaid/chart rendering
160	│ │ ├── prompt_templates.py
161	│ │ ├── callbacks.py # Progress callback helpers
162	│ │ └── ...
163	│ ├── exchange.py # PlanOpticonExchange format
164	│ ├── pipeline.py # Main video processing pipeline
165	│ ├── models.py # Pydantic data models
166	│ └── output_structure.py # Output directory helpers
167	├── tests/ # 822+ tests
168	├── knowledge-base/ # Local-first graph tools
169	│ ├── viewer.html # Self-contained D3.js graph viewer
170	│ └── query.py # Python query script (NetworkX)
171	├── docs/ # MkDocs documentation
172	└── pyproject.toml # Project configuration
173	```
174
175	See [Architecture Overview](architecture/overview.md) for a more detailed breakdown of module responsibilities.
176
177	## Adding a new provider
178
179	Providers self-register via `ProviderRegistry.register()` at module level. When the provider module is imported, it registers itself automatically.
180
181	1. Create `video_processor/providers/your_provider.py`
182	2. Extend `BaseProvider` from `video_processor/providers/base.py`
183	3. Implement the four required methods: `chat()`, `analyze_image()`, `transcribe_audio()`, `list_models()`
184	4. Call `ProviderRegistry.register()` at module level
185	5. Add the import to `video_processor/providers/manager.py` in the lazy-import block
186	6. Add tests in `tests/test_providers.py`
187
188	### Example provider skeleton
189
190	```python
191	"""Your provider implementation."""
192
193	from video_processor.providers.base import BaseProvider, ModelInfo, ProviderRegistry
194
195
196	class YourProvider(BaseProvider):
197	provider_name = "yourprovider"
198
199	def __init__(self, api_key: str \| None = None):
200	import os
201	self.api_key = api_key or os.environ.get("YOUR_API_KEY", "")
202
203	def chat(self, messages, max_tokens=4096, temperature=0.7, model=None):
204	# Implement chat completion
205	...
206
207	def analyze_image(self, image_bytes, prompt, max_tokens=4096, model=None):
208	# Implement image analysis
209	...
210
211	def transcribe_audio(self, audio_path, language=None, model=None):
212	# Implement audio transcription (or raise NotImplementedError)
213	...
214
215	def list_models(self):
216	return [ModelInfo(id="your-model", provider="yourprovider", capabilities=["chat"])]
217
218
219	# Self-registration at import time
220	ProviderRegistry.register(
221	"yourprovider",
222	YourProvider,
223	env_var="YOUR_API_KEY",
224	model_prefixes=["your-"],
225	default_models={"chat": "your-model"},
226	)
227	```
228
229	### OpenAI-compatible providers
230
231	For providers that use the OpenAI API format, extend `OpenAICompatibleProvider` instead of `BaseProvider`. This provides default implementations of `chat()`, `analyze_image()`, and `list_models()` -- you only need to configure the base URL and model mappings.
232
233	```python
234	from video_processor.providers.base import OpenAICompatibleProvider, ProviderRegistry
235
236	class YourProvider(OpenAICompatibleProvider):
237	provider_name = "yourprovider"
238	base_url = "https://api.yourprovider.com/v1"
239	env_var = "YOUR_API_KEY"
240
241	ProviderRegistry.register("yourprovider", YourProvider, env_var="YOUR_API_KEY")
242	```
243
244	## Adding a new cloud source
245
246	Source connectors implement the `BaseSource` ABC from `video_processor/sources/base.py`. Authentication is handled per-source, typically via environment variables.
247
248	1. Create `video_processor/sources/your_source.py`
249	2. Extend `BaseSource`
250	3. Implement `authenticate()`, `list_videos()`, and `download()`
251	4. Add the class to the lazy-import map in `video_processor/sources/__init__.py`
252	5. Add CLI commands in `video_processor/cli/commands.py` if needed
253	6. Add tests and documentation
254
255	### Example source skeleton
256
257	```python
258	"""Your source integration."""
259
260	import os
261	import logging
262	from pathlib import Path
263	from typing import List, Optional
264
265	from video_processor.sources.base import BaseSource, SourceFile
266
267	logger = logging.getLogger(__name__)
268
269
270	class YourSource(BaseSource):
271	def __init__(self, api_key: Optional[str] = None):
272	self.api_key = api_key or os.environ.get("YOUR_SOURCE_KEY", "")
273
274	def authenticate(self) -> bool:
275	"""Validate credentials. Return True on success."""
276	if not self.api_key:
277	logger.error("API key not set. Set YOUR_SOURCE_KEY env var.")
278	return False
279	# Make a test API call to verify credentials
280	...
281	return True
282
283	def list_videos(
284	self,
285	folder_id: Optional[str] = None,
286	folder_path: Optional[str] = None,
287	patterns: Optional[List[str]] = None,
288	) -> List[SourceFile]:
289	"""List available video files."""
290	...
291
292	def download(self, file: SourceFile, destination: Path) -> Path:
293	"""Download a single file. Return the local path."""
294	destination.parent.mkdir(parents=True, exist_ok=True)
295	# Download file content to destination
296	...
297	return destination
298	```
299
300	### Registering in `__init__.py`
301
302	Add your source to the `__all__` list and the `_lazy_map` dictionary in `video_processor/sources/__init__.py`:
303
304	```python
305	__all__ = [
306	...
307	"YourSource",
308	]
309
310	_lazy_map = {
311	...
312	"YourSource": "video_processor.sources.your_source",
313	}
314	```
315
316	## Adding a new skill
317
318	Agent skills extend the `Skill` ABC from `video_processor/agent/skills/base.py` and self-register via `register_skill()`.
319
320	1. Create `video_processor/agent/skills/your_skill.py`
321	2. Extend `Skill` and set `name` and `description` class attributes
322	3. Implement `execute()` to return an `Artifact`
323	4. Optionally override `can_execute()` for custom precondition checks
324	5. Call `register_skill()` at module level
325	6. Add the import to `video_processor/agent/skills/__init__.py`
326	7. Add tests
327
328	### Example skill skeleton
329
330	```python
331	"""Your custom skill."""
332
333	from video_processor.agent.skills.base import AgentContext, Artifact, Skill, register_skill
334
335
336	class YourSkill(Skill):
337	name = "your_skill"
338	description = "Generates a custom artifact from the knowledge graph."
339
340	def execute(self, context: AgentContext, **kwargs) -> Artifact:
341	"""Generate the artifact."""
342	kg_data = context.knowledge_graph.to_dict()
343	# Build content from knowledge graph data
344	content = f"# Your Artifact\n\n{len(kg_data.get('entities', []))} entities found."
345	return Artifact(
346	name="your_artifact",
347	content=content,
348	artifact_type="document",
349	format="markdown",
350	)
351
352	def can_execute(self, context: AgentContext) -> bool:
353	"""Check prerequisites (default requires KG + provider)."""
354	return context.knowledge_graph is not None
355
356
357	# Self-registration at import time
358	register_skill(YourSkill())
359	```
360
361	### Registering in `__init__.py`
362
363	Add the import to `video_processor/agent/skills/__init__.py` so the skill is loaded (and self-registered) when the skills package is imported:
364
365	```python
366	from video_processor.agent.skills import (
367	...
368	your_skill, # noqa: F401
369	)
370	```
371
372	## Adding a new document processor
373
374	Document processors extend the `DocumentProcessor` ABC from `video_processor/processors/base.py` and are registered via `register_processor()`.
375
376	1. Create `video_processor/processors/your_processor.py`
377	2. Extend `DocumentProcessor`
378	3. Set `supported_extensions` class attribute
379	4. Implement `process()` (returns `List[DocumentChunk]`) and `can_process()`
380	5. Call `register_processor()` at module level
381	6. Add the import to `video_processor/processors/__init__.py`
382	7. Add tests
383
384	### Example processor skeleton
385
386	```python
387	"""Your document processor."""
388
389	from pathlib import Path
390	from typing import List
391
392	from video_processor.processors.base import (
393	DocumentChunk,
394	DocumentProcessor,
395	register_processor,
396	)
397
398
399	class YourProcessor(DocumentProcessor):
400	supported_extensions = [".xyz", ".abc"]
401
402	def can_process(self, path: Path) -> bool:
403	return path.suffix.lower() in self.supported_extensions
404
405	def process(self, path: Path) -> List[DocumentChunk]:
406	text = path.read_text()
407	# Split into chunks as appropriate for your format
408	return [
409	DocumentChunk(
410	text=text,
411	source_file=str(path),
412	chunk_index=0,
413	metadata={"format": "xyz"},
414	)
415	]
416
417
418	# Self-registration at import time
419	register_processor([".xyz", ".abc"], YourProcessor)
420	```
421
422	### Registering in `__init__.py`
423
424	Add the import to `video_processor/processors/__init__.py`:
425
426	```python
427	from video_processor.processors import (
428	markdown_processor, # noqa: F401, E402
429	pdf_processor, # noqa: F401, E402
430	your_processor, # noqa: F401, E402
431	)
432	```
433
434	## Adding a new exporter
435
436	Exporters live in `video_processor/exporters/` and are typically called from CLI commands. There is no strict ABC for exporters -- they are plain functions that accept knowledge graph data and an output directory.
437
438	1. Create `video_processor/exporters/your_exporter.py`
439	2. Implement one or more export functions that accept KG data (as a dict) and an output path
440	3. Add CLI integration in `video_processor/cli/commands.py` under the `export` group
441	4. Add tests
442
443	### Example exporter skeleton
444
445	```python
446	"""Your exporter."""
447
448	import json
449	from pathlib import Path
450	from typing import List
451
452
453	def export_your_format(kg_data: dict, output_dir: Path) -> List[Path]:
454	"""Export knowledge graph data in your format.
455
456	Args:
457	kg_data: Knowledge graph as a dict (from KnowledgeGraph.to_dict()).
458	output_dir: Directory to write output files.
459
460	Returns:
461	List of created file paths.
462	"""
463	output_dir.mkdir(parents=True, exist_ok=True)
464	created = []
465
466	output_file = output_dir / "export.xyz"
467	output_file.write_text(json.dumps(kg_data, indent=2))
468	created.append(output_file)
469
470	return created
471	```
472
473	### Adding the CLI command
474
475	Add a subcommand under the `export` group in `video_processor/cli/commands.py`:
476
477	```python
478	@export.command("your-format")
479	@click.argument("db_path", type=click.Path(exists=True))
480	@click.option("-o", "--output", type=click.Path(), default=None)
481	def export_your_format_cmd(db_path, output):
482	"""Export knowledge graph in your format."""
483	from video_processor.exporters.your_exporter import export_your_format
484	from video_processor.integrators.knowledge_graph import KnowledgeGraph
485
486	kg = KnowledgeGraph(db_path=Path(db_path))
487	out_dir = Path(output) if output else Path.cwd() / "your-export"
488	created = export_your_format(kg.to_dict(), out_dir)
489	click.echo(f"Exported {len(created)} files to {out_dir}/")
490	```
491
492	## License
493
494	MIT License -- Copyright (c) 2026 CONFLICT LLC. All rights reserved.
495
496	DDED docs/faq.md

A docs/faq.md

+301

		--- a/docs/faq.md
		+++ b/docs/faq.md
		@@ -0,0 +1,301 @@
	1	+# FAQ & Troubleshooting
	2	+
	3	+## Frequently Asked Questions
	4	+
	5	+### Do I need an API key?
	6	+
	7	+You need at least one of:
	8	+
	9	+- Cloud API key: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, or `GEMINI_API_KEY`
	10	+- Local Ollama: Install [Ollama](https://ollama.com), pull a model, and run `ollama serve`
	11	+
	12	+Some features work without any AI provider:
	13	+
	14	+- `planopticon query stats` — direct knowledge graph queries
	15	+- `planopticon query "entities --type person"` — structured entity lookups
	16	+- `planopticon export markdown` — document generation from existing KG (7 document types, no LLM)
	17	+- `planopticon kg inspect` — knowledge graph statistics
	18	+- `planopticon kg convert` — format conversion
	19	+
	20	+### How much does it cost?
	21	+
	22	+PlanOpticon defaults to cheap models to minimize costs:
	23	+
	24	+\| Task \| Default model \| Approximate cost \|
	25	+\|------\|--------------\|-----------------\|
	26	+\| Chat/analysis \| Claude Haiku / GPT-4o-mini \| ~$0.25-0.50 per 1M tokens \|
	27	+\| Vision (diagrams) \| Gemini Flash / GPT-4o-mini \| ~$0.10-0.50 per 1M tokens \|
	28	+\| Transcription \| Local Whisper (free) / Whisper-1 \| $0.006/minute \|
	29	+
	30	+A typical 1-hour meeting costs roughly $0.05-0.15 to process with default models. Use `--provider ollama` for zero cost.
	31	+
	32	+### Can I run fully offline?
	33	+
	34	+Yes. Install Ollama and local Whisper:
	35	+
	36	+```bash
	37	+ollama pull llama3.2
	38	+ollama pull llava
	39	+pip install planopticon[gpu]
	40	+planopticon analyze -i video.mp4 -o ./output --provider ollama
	41	+```
	42	+
	43	+No data leaves your machine.
	44	+
	45	+### What video formats are supported?
	46	+
	47	+Any format FFmpeg can decode:
	48	+
	49	+- MP4, MKV, AVI, MOV, WebM, FLV, WMV, M4V
	50	+- Container formats with common codecs (H.264, H.265, VP8, VP9, AV1)
	51	+
	52	+### What document formats can I ingest?
	53	+
	54	+- PDF — text extraction via pymupdf or pdfplumber
	55	+- Markdown — parsed with heading-based chunking
	56	+- Plain text — paragraph-based chunking with overlap
	57	+
	58	+### How does the knowledge graph work?
	59	+
	60	+PlanOpticon extracts entities (people, technologies, concepts, decisions) and relationships from your content. These are stored in a SQLite database (`knowledge_graph.db`) with zero external dependencies. Entities are automatically classified using a planning taxonomy (goals, requirements, risks, tasks, milestones).
	61	+
	62	+When you process multiple sources, entities are merged using fuzzy name matching (0.85 threshold) with type conflict resolution and provenance tracking.
	63	+
	64	+### Can I use PlanOpticon with my existing Obsidian vault?
	65	+
	66	+Yes, in both directions:
	67	+
	68	+```bash
	69	+# Ingest an Obsidian vault into PlanOpticon
	70	+planopticon ingest ~/Obsidian/MyVault --output ./kb --recursive
	71	+
	72	+# Export PlanOpticon knowledge to an Obsidian vault
	73	+planopticon export obsidian --input ./kb --output ~/Obsidian/PlanOpticon
	74	+```
	75	+
	76	+The Obsidian export produces proper YAML frontmatter, wiki-links (`[[Entity Name]]`), and tag pages.
	77	+
	78	+### How do I add my own AI provider?
	79	+
	80	+Create a provider module, extend `BaseProvider`, and register it:
	81	+
	82	+```python
	83	+from video_processor.providers.base import BaseProvider, ProviderRegistry
	84	+
	85	+class MyProvider(BaseProvider):
	86	+ provider_name = "myprovider"
	87	+
	88	+ def chat(self, messages, max_tokens=4096, temperature=0.7, model=None):
	89	+ # Your implementation
	90	+ ...
	91	+
	92	+ProviderRegistry.register(
	93	+ name="myprovider",
	94	+ provider_class=MyProvider,
	95	+ env_var="MY_PROVIDER_API_KEY",
	96	+ model_prefixes=["my-"],
	97	+ default_models={"chat": "my-model-v1", "vision": "", "audio": ""},
	98	+)
	99	+```
	100	+
	101	+See the [Contributing guide](contributing.md) for details.
	102	+
	103	+---
	104	+
	105	+## Troubleshooting
	106	+
	107	+### Authentication errors
	108	+
	109	+#### "No auth method available for zoom"
	110	+
	111	+You need to set credentials before authenticating:
	112	+
	113	+```bash
	114	+export ZOOM_CLIENT_ID="your-client-id"
	115	+export ZOOM_CLIENT_SECRET="your-client-secret"
	116	+planopticon auth zoom
	117	+```
	118	+
	119	+The error message tells you which environment variables to set. Each service requires different credentials — see the [Authentication guide](guide/authentication.md).
	120	+
	121	+#### "Token expired" or "401 Unauthorized"
	122	+
	123	+Your saved token has expired and auto-refresh failed. Re-authenticate:
	124	+
	125	+```bash
	126	+planopticon auth google # or whatever service
	127	+```
	128	+
	129	+To clear a stale token:
	130	+
	131	+```bash
	132	+planopticon auth google --logout
	133	+planopticon auth google
	134	+```
	135	+
	136	+Tokens are stored in `~/.planopticon/{service}_token.json`.
	137	+
	138	+#### OAuth redirect errors
	139	+
	140	+If the browser-based OAuth flow fails, check:
	141	+
	142	+1. Your client ID and secret are correct
	143	+2. The redirect URI in your OAuth app matches PlanOpticon's default (`urn:ietf:wg:oauth:2.0:oob`)
	144	+3. The OAuth app has the required scopes enabled
	145	+
	146	+### Provider errors
	147	+
	148	+#### "ANTHROPIC_API_KEY not set"
	149	+
	150	+Set at least one provider's API key:
	151	+
	152	+```bash
	153	+export OPENAI_API_KEY="sk-..."
	154	+# or
	155	+export ANTHROPIC_API_KEY="sk-ant-..."
	156	+# or
	157	+export GEMINI_API_KEY="AI..."
	158	+```
	159	+
	160	+Or use a `.env` file in your project directory.
	161	+
	162	+#### "Unexpected role system" (Anthropic)
	163	+
	164	+This was a bug in older versions where system messages were passed in the messages array instead of as a top-level parameter. Update to v0.4.0 or later.
	165	+
	166	+#### "Model not found" or "Invalid model"
	167	+
	168	+Check available models:
	169	+
	170	+```bash
	171	+planopticon list-models
	172	+```
	173	+
	174	+Common model name issues:
	175	+- Anthropic: use `claude-haiku-4-5-20251001`, not `claude-haiku`
	176	+- OpenAI: use `gpt-4o-mini`, not `gpt4o-mini`
	177	+
	178	+#### Rate limiting / 429 errors
	179	+
	180	+PlanOpticon doesn't currently implement automatic retry. If you hit rate limits:
	181	+
	182	+1. Use a different provider: `--provider gemini`
	183	+2. Use cheaper/faster models: `--chat-model gpt-4o-mini`
	184	+3. Reduce processing depth: `--depth basic`
	185	+4. Use Ollama for zero rate limits: `--provider ollama`
	186	+
	187	+### Processing errors
	188	+
	189	+#### "FFmpeg not found"
	190	+
	191	+Install FFmpeg:
	192	+
	193	+```bash
	194	+# macOS
	195	+brew install ffmpeg
	196	+
	197	+# Ubuntu/Debian
	198	+sudo apt-get install ffmpeg libsndfile1
	199	+
	200	+# Windows
	201	+# Download from https://ffmpeg.org/download.html and add to PATH
	202	+```
	203	+
	204	+#### "Audio extraction failed: no audio track found"
	205	+
	206	+The video file has no audio track. PlanOpticon will skip transcription and continue with frame analysis only.
	207	+
	208	+#### "Frame extraction memory error"
	209	+
	210	+For very long videos, frame extraction can use significant memory. Use the `--max-memory-mb` safety valve:
	211	+
	212	+```bash
	213	+planopticon analyze -i long-video.mp4 -o ./output --max-memory-mb 2048
	214	+```
	215	+
	216	+Or reduce the sampling rate:
	217	+
	218	+```bash
	219	+planopticon analyze -i long-video.mp4 -o ./output --sampling-rate 0.25
	220	+```
	221	+
	222	+#### Batch processing — one video fails
	223	+
	224	+Individual video failures don't stop the batch. Failed videos are logged in the batch manifest with error details. Check `batch_manifest.json` for the specific error.
	225	+
	226	+### Knowledge graph issues
	227	+
	228	+#### "No knowledge graph loaded" in companion
	229	+
	230	+The companion auto-discovers knowledge graphs by looking for `knowledge_graph.db` or `knowledge_graph.json` in the current directory and parent directories. Either:
	231	+
	232	+1. `cd` to the directory containing your knowledge graph
	233	+2. Specify the path explicitly: `planopticon companion --kb ./path/to/kb`
	234	+
	235	+#### Empty or sparse knowledge graph
	236	+
	237	+Common causes:
	238	+
	239	+1. Too few entities extracted: Try `--depth comprehensive` for deeper analysis
	240	+2. Short or low-quality transcript: Check `transcript/transcript.txt` — poor audio produces poor transcription
	241	+3. Wrong provider: Some models extract entities better than others. Try `--provider openai --chat-model gpt-4o` for higher quality
	242	+
	243	+#### Duplicate entities after merge
	244	+
	245	+The fuzzy matching threshold is 0.85 (SequenceMatcher ratio). If you're getting duplicates, the names are too different for automatic matching. You can manually inspect and merge:
	246	+
	247	+```bash
	248	+planopticon kg inspect ./knowledge_graph.db
	249	+planopticon query "entities --name python"
	250	+```
	251	+
	252	+### Companion / REPL issues
	253	+
	254	+#### Chat gives generic advice instead of project-specific answers
	255	+
	256	+The companion needs both a knowledge graph and an LLM provider. Check:
	257	+
	258	+```
	259	+planopticon> /status
	260	+```
	261	+
	262	+If it says "KG: not loaded" or "Provider: none", fix those first:
	263	+
	264	+```
	265	+planopticon> /provider openai
	266	+planopticon> /model gpt-4o-mini
	267	+```
	268	+
	269	+#### Companion is slow
	270	+
	271	+The companion makes LLM API calls for chat messages. To speed things up:
	272	+
	273	+1. Use a faster model: `/model gpt-4o-mini` or `/model claude-haiku-4-5-20251001`
	274	+2. Use direct queries instead of chat: `/entities`, `/search`, `/neighbors` don't need an LLM
	275	+3. Use Ollama locally for lower latency: `/provider ollama`
	276	+
	277	+### Export issues
	278	+
	279	+#### Obsidian export has broken links
	280	+
	281	+Make sure your Obsidian vault has wiki-links enabled (Settings > Files & Links > Use [[Wikilinks]]). PlanOpticon exports use wiki-link syntax by default.
	282	+
	283	+#### PDF export fails
	284	+
	285	+PDF export requires the `pdf` extra:
	286	+
	287	+```bash
	288	+pip install planopticon[pdf]
	289	+```
	290	+
	291	+This installs WeasyPrint, which has system dependencies. On macOS:
	292	+
	293	+```bash
	294	+brew install pango
	295	+```
	296	+
	297	+On Ubuntu:
	298	+
	299	+```bash
	300	+sudo apt-get install libpango1.0-dev
	301	+```

	--- a/docs/faq.md
	+++ b/docs/faq.md
	@@ -0,0 +1,301 @@

	--- a/docs/faq.md
	+++ b/docs/faq.md
	@@ -0,0 +1,301 @@
1	# FAQ & Troubleshooting
2
3	## Frequently Asked Questions
4
5	### Do I need an API key?
6
7	You need at least one of:
8
9	- Cloud API key: `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, or `GEMINI_API_KEY`
10	- Local Ollama: Install [Ollama](https://ollama.com), pull a model, and run `ollama serve`
11
12	Some features work without any AI provider:
13
14	- `planopticon query stats` — direct knowledge graph queries
15	- `planopticon query "entities --type person"` — structured entity lookups
16	- `planopticon export markdown` — document generation from existing KG (7 document types, no LLM)
17	- `planopticon kg inspect` — knowledge graph statistics
18	- `planopticon kg convert` — format conversion
19
20	### How much does it cost?
21
22	PlanOpticon defaults to cheap models to minimize costs:
23
24	\| Task \| Default model \| Approximate cost \|
25	\|------\|--------------\|-----------------\|
26	\| Chat/analysis \| Claude Haiku / GPT-4o-mini \| ~$0.25-0.50 per 1M tokens \|
27	\| Vision (diagrams) \| Gemini Flash / GPT-4o-mini \| ~$0.10-0.50 per 1M tokens \|
28	\| Transcription \| Local Whisper (free) / Whisper-1 \| $0.006/minute \|
29
30	A typical 1-hour meeting costs roughly $0.05-0.15 to process with default models. Use `--provider ollama` for zero cost.
31
32	### Can I run fully offline?
33
34	Yes. Install Ollama and local Whisper:
35
36	```bash
37	ollama pull llama3.2
38	ollama pull llava
39	pip install planopticon[gpu]
40	planopticon analyze -i video.mp4 -o ./output --provider ollama
41	```
42
43	No data leaves your machine.
44
45	### What video formats are supported?
46
47	Any format FFmpeg can decode:
48
49	- MP4, MKV, AVI, MOV, WebM, FLV, WMV, M4V
50	- Container formats with common codecs (H.264, H.265, VP8, VP9, AV1)
51
52	### What document formats can I ingest?
53
54	- PDF — text extraction via pymupdf or pdfplumber
55	- Markdown — parsed with heading-based chunking
56	- Plain text — paragraph-based chunking with overlap
57
58	### How does the knowledge graph work?
59
60	PlanOpticon extracts entities (people, technologies, concepts, decisions) and relationships from your content. These are stored in a SQLite database (`knowledge_graph.db`) with zero external dependencies. Entities are automatically classified using a planning taxonomy (goals, requirements, risks, tasks, milestones).
61
62	When you process multiple sources, entities are merged using fuzzy name matching (0.85 threshold) with type conflict resolution and provenance tracking.
63
64	### Can I use PlanOpticon with my existing Obsidian vault?
65
66	Yes, in both directions:
67
68	```bash
69	# Ingest an Obsidian vault into PlanOpticon
70	planopticon ingest ~/Obsidian/MyVault --output ./kb --recursive
71
72	# Export PlanOpticon knowledge to an Obsidian vault
73	planopticon export obsidian --input ./kb --output ~/Obsidian/PlanOpticon
74	```
75
76	The Obsidian export produces proper YAML frontmatter, wiki-links (`[[Entity Name]]`), and tag pages.
77
78	### How do I add my own AI provider?
79
80	Create a provider module, extend `BaseProvider`, and register it:
81
82	```python
83	from video_processor.providers.base import BaseProvider, ProviderRegistry
84
85	class MyProvider(BaseProvider):
86	provider_name = "myprovider"
87
88	def chat(self, messages, max_tokens=4096, temperature=0.7, model=None):
89	# Your implementation
90	...
91
92	ProviderRegistry.register(
93	name="myprovider",
94	provider_class=MyProvider,
95	env_var="MY_PROVIDER_API_KEY",
96	model_prefixes=["my-"],
97	default_models={"chat": "my-model-v1", "vision": "", "audio": ""},
98	)
99	```
100
101	See the [Contributing guide](contributing.md) for details.
102
103	---
104
105	## Troubleshooting
106
107	### Authentication errors
108
109	#### "No auth method available for zoom"
110
111	You need to set credentials before authenticating:
112
113	```bash
114	export ZOOM_CLIENT_ID="your-client-id"
115	export ZOOM_CLIENT_SECRET="your-client-secret"
116	planopticon auth zoom
117	```
118
119	The error message tells you which environment variables to set. Each service requires different credentials — see the [Authentication guide](guide/authentication.md).
120
121	#### "Token expired" or "401 Unauthorized"
122
123	Your saved token has expired and auto-refresh failed. Re-authenticate:
124
125	```bash
126	planopticon auth google # or whatever service
127	```
128
129	To clear a stale token:
130
131	```bash
132	planopticon auth google --logout
133	planopticon auth google
134	```
135
136	Tokens are stored in `~/.planopticon/{service}_token.json`.
137
138	#### OAuth redirect errors
139
140	If the browser-based OAuth flow fails, check:
141
142	1. Your client ID and secret are correct
143	2. The redirect URI in your OAuth app matches PlanOpticon's default (`urn:ietf:wg:oauth:2.0:oob`)
144	3. The OAuth app has the required scopes enabled
145
146	### Provider errors
147
148	#### "ANTHROPIC_API_KEY not set"
149
150	Set at least one provider's API key:
151
152	```bash
153	export OPENAI_API_KEY="sk-..."
154	# or
155	export ANTHROPIC_API_KEY="sk-ant-..."
156	# or
157	export GEMINI_API_KEY="AI..."
158	```
159
160	Or use a `.env` file in your project directory.
161
162	#### "Unexpected role system" (Anthropic)
163
164	This was a bug in older versions where system messages were passed in the messages array instead of as a top-level parameter. Update to v0.4.0 or later.
165
166	#### "Model not found" or "Invalid model"
167
168	Check available models:
169
170	```bash
171	planopticon list-models
172	```
173
174	Common model name issues:
175	- Anthropic: use `claude-haiku-4-5-20251001`, not `claude-haiku`
176	- OpenAI: use `gpt-4o-mini`, not `gpt4o-mini`
177
178	#### Rate limiting / 429 errors
179
180	PlanOpticon doesn't currently implement automatic retry. If you hit rate limits:
181
182	1. Use a different provider: `--provider gemini`
183	2. Use cheaper/faster models: `--chat-model gpt-4o-mini`
184	3. Reduce processing depth: `--depth basic`
185	4. Use Ollama for zero rate limits: `--provider ollama`
186
187	### Processing errors
188
189	#### "FFmpeg not found"
190
191	Install FFmpeg:
192
193	```bash
194	# macOS
195	brew install ffmpeg
196
197	# Ubuntu/Debian
198	sudo apt-get install ffmpeg libsndfile1
199
200	# Windows
201	# Download from https://ffmpeg.org/download.html and add to PATH
202	```
203
204	#### "Audio extraction failed: no audio track found"
205
206	The video file has no audio track. PlanOpticon will skip transcription and continue with frame analysis only.
207
208	#### "Frame extraction memory error"
209
210	For very long videos, frame extraction can use significant memory. Use the `--max-memory-mb` safety valve:
211
212	```bash
213	planopticon analyze -i long-video.mp4 -o ./output --max-memory-mb 2048
214	```
215
216	Or reduce the sampling rate:
217
218	```bash
219	planopticon analyze -i long-video.mp4 -o ./output --sampling-rate 0.25
220	```
221
222	#### Batch processing — one video fails
223
224	Individual video failures don't stop the batch. Failed videos are logged in the batch manifest with error details. Check `batch_manifest.json` for the specific error.
225
226	### Knowledge graph issues
227
228	#### "No knowledge graph loaded" in companion
229
230	The companion auto-discovers knowledge graphs by looking for `knowledge_graph.db` or `knowledge_graph.json` in the current directory and parent directories. Either:
231
232	1. `cd` to the directory containing your knowledge graph
233	2. Specify the path explicitly: `planopticon companion --kb ./path/to/kb`
234
235	#### Empty or sparse knowledge graph
236
237	Common causes:
238
239	1. Too few entities extracted: Try `--depth comprehensive` for deeper analysis
240	2. Short or low-quality transcript: Check `transcript/transcript.txt` — poor audio produces poor transcription
241	3. Wrong provider: Some models extract entities better than others. Try `--provider openai --chat-model gpt-4o` for higher quality
242
243	#### Duplicate entities after merge
244
245	The fuzzy matching threshold is 0.85 (SequenceMatcher ratio). If you're getting duplicates, the names are too different for automatic matching. You can manually inspect and merge:
246
247	```bash
248	planopticon kg inspect ./knowledge_graph.db
249	planopticon query "entities --name python"
250	```
251
252	### Companion / REPL issues
253
254	#### Chat gives generic advice instead of project-specific answers
255
256	The companion needs both a knowledge graph and an LLM provider. Check:
257
258	```
259	planopticon> /status
260	```
261
262	If it says "KG: not loaded" or "Provider: none", fix those first:
263
264	```
265	planopticon> /provider openai
266	planopticon> /model gpt-4o-mini
267	```
268
269	#### Companion is slow
270
271	The companion makes LLM API calls for chat messages. To speed things up:
272
273	1. Use a faster model: `/model gpt-4o-mini` or `/model claude-haiku-4-5-20251001`
274	2. Use direct queries instead of chat: `/entities`, `/search`, `/neighbors` don't need an LLM
275	3. Use Ollama locally for lower latency: `/provider ollama`
276
277	### Export issues
278
279	#### Obsidian export has broken links
280
281	Make sure your Obsidian vault has wiki-links enabled (Settings > Files & Links > Use [[Wikilinks]]). PlanOpticon exports use wiki-link syntax by default.
282
283	#### PDF export fails
284
285	PDF export requires the `pdf` extra:
286
287	```bash
288	pip install planopticon[pdf]
289	```
290
291	This installs WeasyPrint, which has system dependencies. On macOS:
292
293	```bash
294	brew install pango
295	```
296
297	On Ubuntu:
298
299	```bash
300	sudo apt-get install libpango1.0-dev
301	```

M docs/getting-started/configuration.md

+141 -25

		--- docs/getting-started/configuration.md
		+++ docs/getting-started/configuration.md
		@@ -1,45 +1,150 @@
1	1	# Configuration
2	2
3		-## Environment variables
	3	+## Example `.env` file
	4	+
	5	+Create a `.env` file in your project directory. PlanOpticon loads it automatically.
	6	+
	7	+```bash
	8	+# =============================================================================
	9	+# PlanOpticon Configuration
	10	+# =============================================================================
	11	+# Copy this file to .env and fill in the values you need.
	12	+# You only need ONE AI provider — PlanOpticon auto-detects which are available.
	13	+
	14	+# --- AI Providers (set at least one) ----------------------------------------
	15	+
	16	+# OpenAI — get your key at https://platform.openai.com/api-keys
	17	+OPENAI_API_KEY=sk-...
	18	+
	19	+# Anthropic — get your key at https://console.anthropic.com/settings/keys
	20	+ANTHROPIC_API_KEY=sk-ant-...
	21	+
	22	+# Google Gemini — get your key at https://aistudio.google.com/apikey
	23	+GEMINI_API_KEY=AI...
	24	+
	25	+# Azure OpenAI — from your Azure portal deployment
	26	+# AZURE_OPENAI_API_KEY=...
	27	+# AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
	28	+
	29	+# Together AI — https://api.together.xyz/settings/api-keys
	30	+# TOGETHER_API_KEY=...
	31	+
	32	+# Fireworks AI — https://fireworks.ai/account/api-keys
	33	+# FIREWORKS_API_KEY=...
	34	+
	35	+# Cerebras — https://cloud.cerebras.ai/
	36	+# CEREBRAS_API_KEY=...
	37	+
	38	+# xAI (Grok) — https://console.x.ai/
	39	+# XAI_API_KEY=...
	40	+
	41	+# Ollama (local, no key needed) — just run: ollama serve
	42	+# OLLAMA_HOST=http://localhost:11434
	43	+
	44	+# --- Google (Drive, Docs, Sheets, Meet, YouTube) ----------------------------
	45	+# Option A: OAuth (interactive, recommended for personal use)
	46	+# Create credentials at https://console.cloud.google.com/apis/credentials
	47	+# 1. Create an OAuth 2.0 Client ID (Desktop application)
	48	+# 2. Enable these APIs: Google Drive API, Google Docs API
	49	+GOOGLE_CLIENT_ID=123456789-abc.apps.googleusercontent.com
	50	+GOOGLE_CLIENT_SECRET=GOCSPX-...
	51	+
	52	+# Option B: Service Account (automated/server-side)
	53	+# GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
	54	+
	55	+# --- Zoom (recordings) ------------------------------------------------------
	56	+# Create an OAuth app at https://marketplace.zoom.us/develop/create
	57	+# App type: "General App" with OAuth
	58	+# Scopes: cloud_recording:read:list_user_recordings, cloud_recording:read:recording
	59	+ZOOM_CLIENT_ID=...
	60	+ZOOM_CLIENT_SECRET=...
	61	+# For Server-to-Server (no browser needed):
	62	+# ZOOM_ACCOUNT_ID=...
	63	+
	64	+# --- Microsoft 365 (OneDrive, SharePoint, Teams) ----------------------------
	65	+# Register an app at https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps
	66	+# API permissions: OnlineMeetings.Read, Files.Read (delegated)
	67	+MICROSOFT_CLIENT_ID=...
	68	+MICROSOFT_CLIENT_SECRET=...
	69	+
	70	+# --- Notion ------------------------------------------------------------------
	71	+# Option A: OAuth (create integration at https://www.notion.so/my-integrations)
	72	+# NOTION_CLIENT_ID=...
	73	+# NOTION_CLIENT_SECRET=...
	74	+
	75	+# Option B: API key (simpler, from the same integrations page)
	76	+NOTION_API_KEY=secret_...
	77	+
	78	+# --- GitHub ------------------------------------------------------------------
	79	+# Option A: Personal Access Token (simplest)
	80	+# Create at https://github.com/settings/tokens — needs 'repo' scope
	81	+GITHUB_TOKEN=ghp_...
	82	+
	83	+# Option B: OAuth App (for CI/automation)
	84	+# GITHUB_CLIENT_ID=...
	85	+# GITHUB_CLIENT_SECRET=...
	86	+
	87	+# --- Dropbox -----------------------------------------------------------------
	88	+# Create an app at https://www.dropbox.com/developers/apps
	89	+# DROPBOX_APP_KEY=...
	90	+# DROPBOX_APP_SECRET=...
	91	+# Or use a long-lived access token:
	92	+# DROPBOX_ACCESS_TOKEN=...
	93	+
	94	+# --- General -----------------------------------------------------------------
	95	+# CACHE_DIR=~/.cache/planopticon
	96	+```
	97	+
	98	+## Environment variables reference
4	99
5	100	### AI providers
6	101
7		-\| Variable \| Description \|
8		-\|----------\|-------------\|
9		-\| `OPENAI_API_KEY` \| OpenAI API key \|
10		-\| `ANTHROPIC_API_KEY` \| Anthropic API key \|
11		-\| `GEMINI_API_KEY` \| Google Gemini API key \|
12		-\| `AZURE_OPENAI_API_KEY` \| Azure OpenAI API key \|
13		-\| `AZURE_OPENAI_ENDPOINT` \| Azure OpenAI endpoint URL \|
14		-\| `TOGETHER_API_KEY` \| Together AI API key \|
15		-\| `FIREWORKS_API_KEY` \| Fireworks AI API key \|
16		-\| `CEREBRAS_API_KEY` \| Cerebras API key \|
17		-\| `XAI_API_KEY` \| xAI (Grok) API key \|
18		-\| `OLLAMA_HOST` \| Ollama server URL (default: `http://localhost:11434`) \|
	102	+\| Variable \| Required \| Where to get it \|
	103	+\|----------\|----------\|----------------\|
	104	+\| `OPENAI_API_KEY` \| At least one provider \| [platform.openai.com/api-keys](https://platform.openai.com/api-keys) \|
	105	+\| `ANTHROPIC_API_KEY` \| At least one provider \| [console.anthropic.com](https://console.anthropic.com/settings/keys) \|
	106	+\| `GEMINI_API_KEY` \| At least one provider \| [aistudio.google.com/apikey](https://aistudio.google.com/apikey) \|
	107	+\| `AZURE_OPENAI_API_KEY` \| Optional \| Azure portal > your OpenAI resource \|
	108	+\| `AZURE_OPENAI_ENDPOINT` \| With Azure \| Azure portal > your OpenAI resource \|
	109	+\| `TOGETHER_API_KEY` \| Optional \| [api.together.xyz](https://api.together.xyz/settings/api-keys) \|
	110	+\| `FIREWORKS_API_KEY` \| Optional \| [fireworks.ai](https://fireworks.ai/account/api-keys) \|
	111	+\| `CEREBRAS_API_KEY` \| Optional \| [cloud.cerebras.ai](https://cloud.cerebras.ai/) \|
	112	+\| `XAI_API_KEY` \| Optional \| [console.x.ai](https://console.x.ai/) \|
	113	+\| `OLLAMA_HOST` \| Optional \| Default: `http://localhost:11434` \|
19	114
20	115	### Cloud services
21	116
22		-\| Variable \| Description \|
23		-\|----------\|-------------\|
24		-\| `GOOGLE_APPLICATION_CREDENTIALS` \| Path to Google service account JSON (for server-side Drive access) \|
25		-\| `ZOOM_CLIENT_ID` \| Zoom OAuth app client ID \|
26		-\| `ZOOM_CLIENT_SECRET` \| Zoom OAuth app client secret \|
27		-\| `NOTION_API_KEY` \| Notion integration token \|
28		-\| `GITHUB_TOKEN` \| GitHub personal access token \|
29		-\| `MICROSOFT_CLIENT_ID` \| Azure AD app client ID (for Microsoft 365) \|
30		-\| `MICROSOFT_CLIENT_SECRET` \| Azure AD app client secret \|
	117	+\| Variable \| Service \| Auth method \|
	118	+\|----------\|---------\|-------------\|
	119	+\| `GOOGLE_CLIENT_ID` \| Google (Drive, Docs, Meet) \| OAuth \|
	120	+\| `GOOGLE_CLIENT_SECRET` \| Google \| OAuth \|
	121	+\| `GOOGLE_APPLICATION_CREDENTIALS` \| Google \| Service account \|
	122	+\| `ZOOM_CLIENT_ID` \| Zoom \| OAuth \|
	123	+\| `ZOOM_CLIENT_SECRET` \| Zoom \| OAuth \|
	124	+\| `ZOOM_ACCOUNT_ID` \| Zoom \| Server-to-Server \|
	125	+\| `MICROSOFT_CLIENT_ID` \| Microsoft 365 \| OAuth \|
	126	+\| `MICROSOFT_CLIENT_SECRET` \| Microsoft 365 \| OAuth \|
	127	+\| `NOTION_CLIENT_ID` \| Notion \| OAuth \|
	128	+\| `NOTION_CLIENT_SECRET` \| Notion \| OAuth \|
	129	+\| `NOTION_API_KEY` \| Notion \| API key \|
	130	+\| `GITHUB_CLIENT_ID` \| GitHub \| OAuth \|
	131	+\| `GITHUB_CLIENT_SECRET` \| GitHub \| OAuth \|
	132	+\| `GITHUB_TOKEN` \| GitHub \| API key \|
	133	+\| `DROPBOX_APP_KEY` \| Dropbox \| OAuth \|
	134	+\| `DROPBOX_APP_SECRET` \| Dropbox \| OAuth \|
	135	+\| `DROPBOX_ACCESS_TOKEN` \| Dropbox \| API key \|
31	136
32	137	### General
33	138
34	139	\| Variable \| Description \|
35	140	\|----------\|-------------\|
36	141	\| `CACHE_DIR` \| Directory for API response caching \|
37	142
38	143	## Authentication
39	144
40		-Most cloud services use OAuth via the `planopticon auth` command. Run it once per service to store credentials locally:
	145	+PlanOpticon uses OAuth for cloud services. Run `planopticon auth` once per service — tokens are saved locally and refreshed automatically.
41	146
42	147	```bash
43	148	planopticon auth google # Google Drive, Docs, Meet, YouTube
44	149	planopticon auth dropbox # Dropbox
45	150	planopticon auth zoom # Zoom recordings
		@@ -46,13 +151,24 @@
46	151	planopticon auth notion # Notion pages
47	152	planopticon auth github # GitHub repos and wikis
48	153	planopticon auth microsoft # OneDrive, SharePoint, Teams
49	154	```
50	155
51		-Credentials are stored in `~/.config/planopticon/`. Use `planopticon auth SERVICE --logout` to remove them.
	156	+Credentials are stored in `~/.planopticon/`. Use `planopticon auth SERVICE --logout` to remove them.
	157	+
	158	+### What each service needs
	159	+
	160	+\| Service \| Minimum setup \| Full OAuth setup \|
	161	+\|---------\|--------------\|-----------------\|
	162	+\| Google \| `GOOGLE_CLIENT_ID` + `GOOGLE_CLIENT_SECRET` \| Create OAuth credentials in [Google Cloud Console](https://console.cloud.google.com/apis/credentials) \|
	163	+\| Zoom \| `ZOOM_CLIENT_ID` + `ZOOM_CLIENT_SECRET` \| Create a General App at [marketplace.zoom.us](https://marketplace.zoom.us/develop/create) \|
	164	+\| Microsoft \| `MICROSOFT_CLIENT_ID` + `MICROSOFT_CLIENT_SECRET` \| Register app in [Azure AD](https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps) \|
	165	+\| Notion \| `NOTION_API_KEY` (simplest) \| Create integration at [notion.so/my-integrations](https://www.notion.so/my-integrations) \|
	166	+\| GitHub \| `GITHUB_TOKEN` (simplest) \| Create token at [github.com/settings/tokens](https://github.com/settings/tokens) \|
	167	+\| Dropbox \| `DROPBOX_APP_KEY` + `DROPBOX_APP_SECRET` \| Create app at [dropbox.com/developers](https://www.dropbox.com/developers/apps) \|
52	168
53		-For Zoom and Microsoft 365, you also need to set the client ID and secret environment variables before running `planopticon auth`.
	169	+For detailed OAuth app creation walkthroughs, see the [Authentication guide](../guide/authentication.md).
54	170
55	171	## Provider routing
56	172
57	173	PlanOpticon auto-discovers available models and routes each task to the cheapest capable option:
58	174
59	175
60	176	ADDED docs/guide/authentication.md

	--- docs/getting-started/configuration.md
	+++ docs/getting-started/configuration.md
	@@ -1,45 +1,150 @@
1	# Configuration
2
3	## Environment variables































































































4
5	### AI providers
6
7	\| Variable \| Description \|
8	\|----------\|-------------\|
9	\| `OPENAI_API_KEY` \| OpenAI API key \|
10	\| `ANTHROPIC_API_KEY` \| Anthropic API key \|
11	\| `GEMINI_API_KEY` \| Google Gemini API key \|
12	\| `AZURE_OPENAI_API_KEY` \| Azure OpenAI API key \|
13	\| `AZURE_OPENAI_ENDPOINT` \| Azure OpenAI endpoint URL \|
14	\| `TOGETHER_API_KEY` \| Together AI API key \|
15	\| `FIREWORKS_API_KEY` \| Fireworks AI API key \|
16	\| `CEREBRAS_API_KEY` \| Cerebras API key \|
17	\| `XAI_API_KEY` \| xAI (Grok) API key \|
18	\| `OLLAMA_HOST` \| Ollama server URL (default: `http://localhost:11434`) \|
19
20	### Cloud services
21
22	\| Variable \| Description \|
23	\|----------\|-------------\|
24	\| `GOOGLE_APPLICATION_CREDENTIALS` \| Path to Google service account JSON (for server-side Drive access) \|
25	\| `ZOOM_CLIENT_ID` \| Zoom OAuth app client ID \|
26	\| `ZOOM_CLIENT_SECRET` \| Zoom OAuth app client secret \|
27	\| `NOTION_API_KEY` \| Notion integration token \|
28	\| `GITHUB_TOKEN` \| GitHub personal access token \|
29	\| `MICROSOFT_CLIENT_ID` \| Azure AD app client ID (for Microsoft 365) \|
30	\| `MICROSOFT_CLIENT_SECRET` \| Azure AD app client secret \|










31
32	### General
33
34	\| Variable \| Description \|
35	\|----------\|-------------\|
36	\| `CACHE_DIR` \| Directory for API response caching \|
37
38	## Authentication
39
40	Most cloud services use OAuth via the `planopticon auth` command. Run it once per service to store credentials locally:
41
42	```bash
43	planopticon auth google # Google Drive, Docs, Meet, YouTube
44	planopticon auth dropbox # Dropbox
45	planopticon auth zoom # Zoom recordings
	@@ -46,13 +151,24 @@
46	planopticon auth notion # Notion pages
47	planopticon auth github # GitHub repos and wikis
48	planopticon auth microsoft # OneDrive, SharePoint, Teams
49	```
50
51	Credentials are stored in `~/.config/planopticon/`. Use `planopticon auth SERVICE --logout` to remove them.











52
53	For Zoom and Microsoft 365, you also need to set the client ID and secret environment variables before running `planopticon auth`.
54
55	## Provider routing
56
57	PlanOpticon auto-discovers available models and routes each task to the cheapest capable option:
58
59
60	DDED docs/guide/authentication.md

	--- docs/getting-started/configuration.md
	+++ docs/getting-started/configuration.md
	@@ -1,45 +1,150 @@
1	# Configuration
2
3	## Example `.env` file
4
5	Create a `.env` file in your project directory. PlanOpticon loads it automatically.
6
7	```bash
8	# =============================================================================
9	# PlanOpticon Configuration
10	# =============================================================================
11	# Copy this file to .env and fill in the values you need.
12	# You only need ONE AI provider — PlanOpticon auto-detects which are available.
13
14	# --- AI Providers (set at least one) ----------------------------------------
15
16	# OpenAI — get your key at https://platform.openai.com/api-keys
17	OPENAI_API_KEY=sk-...
18
19	# Anthropic — get your key at https://console.anthropic.com/settings/keys
20	ANTHROPIC_API_KEY=sk-ant-...
21
22	# Google Gemini — get your key at https://aistudio.google.com/apikey
23	GEMINI_API_KEY=AI...
24
25	# Azure OpenAI — from your Azure portal deployment
26	# AZURE_OPENAI_API_KEY=...
27	# AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
28
29	# Together AI — https://api.together.xyz/settings/api-keys
30	# TOGETHER_API_KEY=...
31
32	# Fireworks AI — https://fireworks.ai/account/api-keys
33	# FIREWORKS_API_KEY=...
34
35	# Cerebras — https://cloud.cerebras.ai/
36	# CEREBRAS_API_KEY=...
37
38	# xAI (Grok) — https://console.x.ai/
39	# XAI_API_KEY=...
40
41	# Ollama (local, no key needed) — just run: ollama serve
42	# OLLAMA_HOST=http://localhost:11434
43
44	# --- Google (Drive, Docs, Sheets, Meet, YouTube) ----------------------------
45	# Option A: OAuth (interactive, recommended for personal use)
46	# Create credentials at https://console.cloud.google.com/apis/credentials
47	# 1. Create an OAuth 2.0 Client ID (Desktop application)
48	# 2. Enable these APIs: Google Drive API, Google Docs API
49	GOOGLE_CLIENT_ID=123456789-abc.apps.googleusercontent.com
50	GOOGLE_CLIENT_SECRET=GOCSPX-...
51
52	# Option B: Service Account (automated/server-side)
53	# GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
54
55	# --- Zoom (recordings) ------------------------------------------------------
56	# Create an OAuth app at https://marketplace.zoom.us/develop/create
57	# App type: "General App" with OAuth
58	# Scopes: cloud_recording:read:list_user_recordings, cloud_recording:read:recording
59	ZOOM_CLIENT_ID=...
60	ZOOM_CLIENT_SECRET=...
61	# For Server-to-Server (no browser needed):
62	# ZOOM_ACCOUNT_ID=...
63
64	# --- Microsoft 365 (OneDrive, SharePoint, Teams) ----------------------------
65	# Register an app at https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps
66	# API permissions: OnlineMeetings.Read, Files.Read (delegated)
67	MICROSOFT_CLIENT_ID=...
68	MICROSOFT_CLIENT_SECRET=...
69
70	# --- Notion ------------------------------------------------------------------
71	# Option A: OAuth (create integration at https://www.notion.so/my-integrations)
72	# NOTION_CLIENT_ID=...
73	# NOTION_CLIENT_SECRET=...
74
75	# Option B: API key (simpler, from the same integrations page)
76	NOTION_API_KEY=secret_...
77
78	# --- GitHub ------------------------------------------------------------------
79	# Option A: Personal Access Token (simplest)
80	# Create at https://github.com/settings/tokens — needs 'repo' scope
81	GITHUB_TOKEN=ghp_...
82
83	# Option B: OAuth App (for CI/automation)
84	# GITHUB_CLIENT_ID=...
85	# GITHUB_CLIENT_SECRET=...
86
87	# --- Dropbox -----------------------------------------------------------------
88	# Create an app at https://www.dropbox.com/developers/apps
89	# DROPBOX_APP_KEY=...
90	# DROPBOX_APP_SECRET=...
91	# Or use a long-lived access token:
92	# DROPBOX_ACCESS_TOKEN=...
93
94	# --- General -----------------------------------------------------------------
95	# CACHE_DIR=~/.cache/planopticon
96	```
97
98	## Environment variables reference
99
100	### AI providers
101
102	\| Variable \| Required \| Where to get it \|
103	\|----------\|----------\|----------------\|
104	\| `OPENAI_API_KEY` \| At least one provider \| [platform.openai.com/api-keys](https://platform.openai.com/api-keys) \|
105	\| `ANTHROPIC_API_KEY` \| At least one provider \| [console.anthropic.com](https://console.anthropic.com/settings/keys) \|
106	\| `GEMINI_API_KEY` \| At least one provider \| [aistudio.google.com/apikey](https://aistudio.google.com/apikey) \|
107	\| `AZURE_OPENAI_API_KEY` \| Optional \| Azure portal > your OpenAI resource \|
108	\| `AZURE_OPENAI_ENDPOINT` \| With Azure \| Azure portal > your OpenAI resource \|
109	\| `TOGETHER_API_KEY` \| Optional \| [api.together.xyz](https://api.together.xyz/settings/api-keys) \|
110	\| `FIREWORKS_API_KEY` \| Optional \| [fireworks.ai](https://fireworks.ai/account/api-keys) \|
111	\| `CEREBRAS_API_KEY` \| Optional \| [cloud.cerebras.ai](https://cloud.cerebras.ai/) \|
112	\| `XAI_API_KEY` \| Optional \| [console.x.ai](https://console.x.ai/) \|
113	\| `OLLAMA_HOST` \| Optional \| Default: `http://localhost:11434` \|
114
115	### Cloud services
116
117	\| Variable \| Service \| Auth method \|
118	\|----------\|---------\|-------------\|
119	\| `GOOGLE_CLIENT_ID` \| Google (Drive, Docs, Meet) \| OAuth \|
120	\| `GOOGLE_CLIENT_SECRET` \| Google \| OAuth \|
121	\| `GOOGLE_APPLICATION_CREDENTIALS` \| Google \| Service account \|
122	\| `ZOOM_CLIENT_ID` \| Zoom \| OAuth \|
123	\| `ZOOM_CLIENT_SECRET` \| Zoom \| OAuth \|
124	\| `ZOOM_ACCOUNT_ID` \| Zoom \| Server-to-Server \|
125	\| `MICROSOFT_CLIENT_ID` \| Microsoft 365 \| OAuth \|
126	\| `MICROSOFT_CLIENT_SECRET` \| Microsoft 365 \| OAuth \|
127	\| `NOTION_CLIENT_ID` \| Notion \| OAuth \|
128	\| `NOTION_CLIENT_SECRET` \| Notion \| OAuth \|
129	\| `NOTION_API_KEY` \| Notion \| API key \|
130	\| `GITHUB_CLIENT_ID` \| GitHub \| OAuth \|
131	\| `GITHUB_CLIENT_SECRET` \| GitHub \| OAuth \|
132	\| `GITHUB_TOKEN` \| GitHub \| API key \|
133	\| `DROPBOX_APP_KEY` \| Dropbox \| OAuth \|
134	\| `DROPBOX_APP_SECRET` \| Dropbox \| OAuth \|
135	\| `DROPBOX_ACCESS_TOKEN` \| Dropbox \| API key \|
136
137	### General
138
139	\| Variable \| Description \|
140	\|----------\|-------------\|
141	\| `CACHE_DIR` \| Directory for API response caching \|
142
143	## Authentication
144
145	PlanOpticon uses OAuth for cloud services. Run `planopticon auth` once per service — tokens are saved locally and refreshed automatically.
146
147	```bash
148	planopticon auth google # Google Drive, Docs, Meet, YouTube
149	planopticon auth dropbox # Dropbox
150	planopticon auth zoom # Zoom recordings
	@@ -46,13 +151,24 @@
151	planopticon auth notion # Notion pages
152	planopticon auth github # GitHub repos and wikis
153	planopticon auth microsoft # OneDrive, SharePoint, Teams
154	```
155
156	Credentials are stored in `~/.planopticon/`. Use `planopticon auth SERVICE --logout` to remove them.
157
158	### What each service needs
159
160	\| Service \| Minimum setup \| Full OAuth setup \|
161	\|---------\|--------------\|-----------------\|
162	\| Google \| `GOOGLE_CLIENT_ID` + `GOOGLE_CLIENT_SECRET` \| Create OAuth credentials in [Google Cloud Console](https://console.cloud.google.com/apis/credentials) \|
163	\| Zoom \| `ZOOM_CLIENT_ID` + `ZOOM_CLIENT_SECRET` \| Create a General App at [marketplace.zoom.us](https://marketplace.zoom.us/develop/create) \|
164	\| Microsoft \| `MICROSOFT_CLIENT_ID` + `MICROSOFT_CLIENT_SECRET` \| Register app in [Azure AD](https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps) \|
165	\| Notion \| `NOTION_API_KEY` (simplest) \| Create integration at [notion.so/my-integrations](https://www.notion.so/my-integrations) \|
166	\| GitHub \| `GITHUB_TOKEN` (simplest) \| Create token at [github.com/settings/tokens](https://github.com/settings/tokens) \|
167	\| Dropbox \| `DROPBOX_APP_KEY` + `DROPBOX_APP_SECRET` \| Create app at [dropbox.com/developers](https://www.dropbox.com/developers/apps) \|
168
169	For detailed OAuth app creation walkthroughs, see the [Authentication guide](../guide/authentication.md).
170
171	## Provider routing
172
173	PlanOpticon auto-discovers available models and routes each task to the cheapest capable option:
174
175
176	DDED docs/guide/authentication.md

A docs/guide/authentication.md

+525

		--- a/docs/guide/authentication.md
		+++ b/docs/guide/authentication.md
		@@ -0,0 +1,525 @@
	1	+# Authentication
	2	+
	3	+PlanOpticon uses a unified authentication system to connect with cloud services for fetching recordings, documents, and other content. The system is OAuth-first: it prefers OAuth 2.0 flows for security and token management, but falls back to API keys when OAuth is not configured.
	4	+
	5	+## Auth strategy overview
	6	+
	7	+PlanOpticon supports six cloud services out of the box: Google, Dropbox, Zoom, Notion, GitHub, and Microsoft. Each service uses the same authentication chain, implemented through the `OAuthManager` class. You configure credentials once (via environment variables or directly), and PlanOpticon handles token acquisition, storage, refresh, and fallback automatically.
	8	+
	9	+All authentication state is managed through the `planopticon auth` CLI command, the `/auth` companion REPL command, or programmatically via the Python API.
	10	+
	11	+## The auth chain
	12	+
	13	+When you authenticate with a service, PlanOpticon tries the following methods in order. It stops at the first one that succeeds:
	14	+
	15	+1. Saved token -- Checks `~/.planopticon/{service}_token.json` for a previously saved token. If the token has not expired, it is used immediately. If it has expired but a refresh token is available, PlanOpticon attempts an automatic token refresh.
	16	+
	17	+2. Client Credentials grant (Server-to-Server) -- If an `account_id` is configured (e.g., `ZOOM_ACCOUNT_ID`), PlanOpticon attempts a client credentials grant. This is a non-interactive flow suitable for automated pipelines and server-side integrations. No browser is required.
	18	+
	19	+3. OAuth 2.0 Authorization Code with PKCE (interactive) -- If a client ID is configured and OAuth endpoints are available, PlanOpticon initiates an interactive OAuth PKCE flow. It opens a browser to the service's authorization page, waits for you to paste the authorization code, and exchanges it for tokens. The tokens are saved for future use.
	20	+
	21	+4. API key fallback -- If no OAuth method succeeds, PlanOpticon checks for a service-specific API key environment variable (e.g., `GITHUB_TOKEN`, `NOTION_API_KEY`). This is the simplest setup but may have reduced capabilities compared to OAuth.
	22	+
	23	+If none of the four methods succeed, PlanOpticon returns an error with hints about which environment variables to set.
	24	+
	25	+## Token storage
	26	+
	27	+Tokens are persisted as JSON files in `~/.planopticon/`:
	28	+
	29	+```
	30	+~/.planopticon/
	31	+ google_token.json
	32	+ dropbox_token.json
	33	+ zoom_token.json
	34	+ notion_token.json
	35	+ github_token.json
	36	+ microsoft_token.json
	37	+```
	38	+
	39	+Each token file contains:
	40	+
	41	+\| Field \| Description \|
	42	+\|-------\|-------------\|
	43	+\| `access_token` \| The current access token \|
	44	+\| `refresh_token` \| Refresh token for automatic renewal (if provided by the service) \|
	45	+\| `expires_at` \| Unix timestamp when the token expires (with a 60-second safety margin) \|
	46	+\| `client_id` \| The client ID used for this token (for refresh) \|
	47	+\| `client_secret` \| The client secret used (for refresh) \|
	48	+
	49	+The `~/.planopticon/` directory is created automatically on first use. Token files are overwritten on each successful authentication or refresh.
	50	+
	51	+To remove a saved token, use `planopticon auth <service> --logout` or delete the file directly.
	52	+
	53	+## Supported services
	54	+
	55	+### Google
	56	+
	57	+Google authentication provides access to Google Drive and Google Docs for fetching documents, recordings, and other content.
	58	+
	59	+Scopes requested:
	60	+
	61	+- `https://www.googleapis.com/auth/drive.readonly`
	62	+- `https://www.googleapis.com/auth/documents.readonly`
	63	+
	64	+Environment variables:
	65	+
	66	+\| Variable \| Required \| Description \|
	67	+\|----------\|----------\|-------------\|
	68	+\| `GOOGLE_CLIENT_ID` \| For OAuth \| OAuth 2.0 Client ID from Google Cloud Console \|
	69	+\| `GOOGLE_CLIENT_SECRET` \| For OAuth \| OAuth 2.0 Client Secret \|
	70	+\| `GOOGLE_API_KEY` \| Fallback \| API key (limited access, no user-specific data) \|
	71	+
	72	+OAuth app setup:
	73	+
	74	+1. Go to the [Google Cloud Console](https://console.cloud.google.com/).
	75	+2. Create a project (or select an existing one).
	76	+3. Navigate to APIs & Services > Credentials.
	77	+4. Click Create Credentials > OAuth client ID.
	78	+5. Choose Desktop app as the application type.
	79	+6. Copy the Client ID and Client Secret.
	80	+7. Under APIs & Services > Library, enable the Google Drive API and Google Docs API.
	81	+8. Set the environment variables:
	82	+
	83	+```bash
	84	+export GOOGLE_CLIENT_ID="your-client-id.apps.googleusercontent.com"
	85	+export GOOGLE_CLIENT_SECRET="your-client-secret"
	86	+```
	87	+
	88	+Service account fallback: For automated pipelines, you can use a Google service account instead of OAuth. Generate a service account key JSON file from the Google Cloud Console and set `GOOGLE_APPLICATION_CREDENTIALS` to point to it. The PlanOpticon Google Workspace connector (`planopticon gws`) uses the `gws` CLI which has its own auth flow via `gws auth login`.
	89	+
	90	+### Dropbox
	91	+
	92	+Dropbox authentication provides access to files stored in Dropbox.
	93	+
	94	+Environment variables:
	95	+
	96	+\| Variable \| Required \| Description \|
	97	+\|----------\|----------\|-------------\|
	98	+\| `DROPBOX_APP_KEY` \| For OAuth \| App key from the Dropbox App Console \|
	99	+\| `DROPBOX_APP_SECRET` \| For OAuth \| App secret \|
	100	+\| `DROPBOX_ACCESS_TOKEN` \| Fallback \| Long-lived access token (for quick setup) \|
	101	+
	102	+OAuth app setup:
	103	+
	104	+1. Go to the [Dropbox App Console](https://www.dropbox.com/developers/apps).
	105	+2. Click Create App.
	106	+3. Choose Scoped access and Full Dropbox (or App folder for restricted access).
	107	+4. Copy the App key and App secret from the Settings tab.
	108	+5. Set the environment variables:
	109	+
	110	+```bash
	111	+export DROPBOX_APP_KEY="your-app-key"
	112	+export DROPBOX_APP_SECRET="your-app-secret"
	113	+```
	114	+
	115	+Access token shortcut: For quick testing, you can generate an access token directly from the app's Settings page in the Dropbox App Console and set it as `DROPBOX_ACCESS_TOKEN`. This bypasses OAuth entirely but the token may have a limited lifetime.
	116	+
	117	+### Zoom
	118	+
	119	+Zoom authentication provides access to cloud recordings, meeting metadata, and transcripts.
	120	+
	121	+Environment variables:
	122	+
	123	+\| Variable \| Required \| Description \|
	124	+\|----------\|----------\|-------------\|
	125	+\| `ZOOM_CLIENT_ID` \| For OAuth \| OAuth client ID from the Zoom Marketplace \|
	126	+\| `ZOOM_CLIENT_SECRET` \| For OAuth \| OAuth client secret \|
	127	+\| `ZOOM_ACCOUNT_ID` \| For S2S \| Account ID for Server-to-Server OAuth \|
	128	+
	129	+Server-to-Server (recommended for automation):
	130	+
	131	+When `ZOOM_ACCOUNT_ID` is set alongside `ZOOM_CLIENT_ID` and `ZOOM_CLIENT_SECRET`, PlanOpticon uses the client credentials grant (Server-to-Server OAuth). This is non-interactive and ideal for CI/CD pipelines and scheduled jobs.
	132	+
	133	+1. Go to the [Zoom Marketplace](https://marketplace.zoom.us/).
	134	+2. Click Develop > Build App.
	135	+3. Choose Server-to-Server OAuth.
	136	+4. Copy the Account ID, Client ID, and Client Secret.
	137	+5. Add the required scopes: `recording:read:admin` (or `recording:read`).
	138	+6. Set the environment variables:
	139	+
	140	+```bash
	141	+export ZOOM_CLIENT_ID="your-client-id"
	142	+export ZOOM_CLIENT_SECRET="your-client-secret"
	143	+export ZOOM_ACCOUNT_ID="your-account-id"
	144	+```
	145	+
	146	+User-level OAuth PKCE:
	147	+
	148	+If `ZOOM_ACCOUNT_ID` is not set, PlanOpticon falls back to the interactive OAuth PKCE flow. This opens a browser window for the user to authorize access.
	149	+
	150	+1. In the Zoom Marketplace, create a General App (or OAuth app).
	151	+2. Set the redirect URI to `urn:ietf:wg:oauth:2.0:oob` (out-of-band).
	152	+3. Copy the Client ID and Client Secret.
	153	+
	154	+### Notion
	155	+
	156	+Notion authentication provides access to pages, databases, and content in your Notion workspace.
	157	+
	158	+Environment variables:
	159	+
	160	+\| Variable \| Required \| Description \|
	161	+\|----------\|----------\|-------------\|
	162	+\| `NOTION_CLIENT_ID` \| For OAuth \| OAuth client ID from the Notion Integrations page \|
	163	+\| `NOTION_CLIENT_SECRET` \| For OAuth \| OAuth client secret \|
	164	+\| `NOTION_API_KEY` \| Fallback \| Internal integration token \|
	165	+
	166	+OAuth app setup:
	167	+
	168	+1. Go to [My Integrations](https://www.notion.so/my-integrations) in Notion.
	169	+2. Click New integration.
	170	+3. Select Public integration (required for OAuth).
	171	+4. Copy the OAuth Client ID and Client Secret.
	172	+5. Set the redirect URI.
	173	+6. Set the environment variables:
	174	+
	175	+```bash
	176	+export NOTION_CLIENT_ID="your-client-id"
	177	+export NOTION_CLIENT_SECRET="your-client-secret"
	178	+```
	179	+
	180	+Internal integration (API key fallback):
	181	+
	182	+For simpler setups, create an Internal integration from the Notion Integrations page. Copy the integration token and set it as `NOTION_API_KEY`. You must also share the relevant Notion pages/databases with the integration.
	183	+
	184	+```bash
	185	+export NOTION_API_KEY="ntn_your-integration-token"
	186	+```
	187	+
	188	+### GitHub
	189	+
	190	+GitHub authentication provides access to repositories, issues, and organization data.
	191	+
	192	+Scopes requested:
	193	+
	194	+- `repo`
	195	+- `read:org`
	196	+
	197	+Environment variables:
	198	+
	199	+\| Variable \| Required \| Description \|
	200	+\|----------\|----------\|-------------\|
	201	+\| `GITHUB_CLIENT_ID` \| For OAuth \| OAuth App client ID \|
	202	+\| `GITHUB_CLIENT_SECRET` \| For OAuth \| OAuth App client secret \|
	203	+\| `GITHUB_TOKEN` \| Fallback \| Personal access token (classic or fine-grained) \|
	204	+
	205	+OAuth app setup:
	206	+
	207	+1. Go to GitHub > Settings > Developer Settings > OAuth Apps.
	208	+2. Click New OAuth App.
	209	+3. Set the Authorization callback URL to `urn:ietf:wg:oauth:2.0:oob`.
	210	+4. Copy the Client ID and generate a Client Secret.
	211	+5. Set the environment variables:
	212	+
	213	+```bash
	214	+export GITHUB_CLIENT_ID="your-client-id"
	215	+export GITHUB_CLIENT_SECRET="your-client-secret"
	216	+```
	217	+
	218	+Personal access token (recommended for most users):
	219	+
	220	+The simplest approach is to create a Personal Access Token:
	221	+
	222	+1. Go to GitHub > Settings > Developer Settings > Personal Access Tokens.
	223	+2. Generate a token with `repo` and `read:org` scopes.
	224	+3. Set it as `GITHUB_TOKEN`:
	225	+
	226	+```bash
	227	+export GITHUB_TOKEN="ghp_your-token"
	228	+```
	229	+
	230	+### Microsoft
	231	+
	232	+Microsoft authentication provides access to Microsoft 365 resources via the Microsoft Graph API, including OneDrive, SharePoint, and Teams recordings.
	233	+
	234	+Scopes requested:
	235	+
	236	+- `https://graph.microsoft.com/OnlineMeetings.Read`
	237	+- `https://graph.microsoft.com/Files.Read`
	238	+
	239	+Environment variables:
	240	+
	241	+\| Variable \| Required \| Description \|
	242	+\|----------\|----------\|-------------\|
	243	+\| `MICROSOFT_CLIENT_ID` \| For OAuth \| Application (client) ID from Azure AD \|
	244	+\| `MICROSOFT_CLIENT_SECRET` \| For OAuth \| Client secret from Azure AD \|
	245	+
	246	+Azure AD app registration:
	247	+
	248	+1. Go to the [Azure Portal](https://portal.azure.com/).
	249	+2. Navigate to Azure Active Directory > App registrations.
	250	+3. Click New registration.
	251	+4. Name the application (e.g., "PlanOpticon").
	252	+5. Under Supported account types, select the appropriate option for your organization.
	253	+6. Set the redirect URI to `urn:ietf:wg:oauth:2.0:oob` with platform Mobile and desktop applications.
	254	+7. After registration, go to Certificates & secrets and create a new client secret.
	255	+8. Under API permissions, add:
	256	+ - `OnlineMeetings.Read`
	257	+ - `Files.Read`
	258	+9. Grant admin consent if required by your organization.
	259	+10. Set the environment variables:
	260	+
	261	+```bash
	262	+export MICROSOFT_CLIENT_ID="your-application-id"
	263	+export MICROSOFT_CLIENT_SECRET="your-client-secret"
	264	+```
	265	+
	266	+Microsoft 365 CLI: The `planopticon m365` commands use the `@pnp/cli-microsoft365` npm package, which has its own authentication flow via `m365 login`. This is separate from the OAuth flow described above.
	267	+
	268	+## CLI usage
	269	+
	270	+### `planopticon auth`
	271	+
	272	+Authenticate with a cloud service or manage saved tokens.
	273	+
	274	+```
	275	+planopticon auth SERVICE [--logout]
	276	+```
	277	+
	278	+Arguments:
	279	+
	280	+\| Argument \| Description \|
	281	+\|----------\|-------------\|
	282	+\| `SERVICE` \| One of: `google`, `dropbox`, `zoom`, `notion`, `github`, `microsoft` \|
	283	+
	284	+Options:
	285	+
	286	+\| Option \| Description \|
	287	+\|--------\|-------------\|
	288	+\| `--logout` \| Clear the saved token for the specified service \|
	289	+
	290	+Examples:
	291	+
	292	+```bash
	293	+# Authenticate with Google (triggers OAuth flow or uses saved token)
	294	+planopticon auth google
	295	+
	296	+# Authenticate with Zoom
	297	+planopticon auth zoom
	298	+
	299	+# Clear saved GitHub token
	300	+planopticon auth github --logout
	301	+```
	302	+
	303	+On success, the command prints the authentication method used:
	304	+
	305	+```
	306	+Google authentication successful (oauth_pkce).
	307	+```
	308	+
	309	+or
	310	+
	311	+```
	312	+Github authentication successful (api_key).
	313	+```
	314	+
	315	+### Companion REPL `/auth`
	316	+
	317	+Inside the interactive companion REPL (`planopticon -C` or `planopticon -I`), you can authenticate with services using the `/auth` command:
	318	+
	319	+```
	320	+/auth SERVICE
	321	+```
	322	+
	323	+Without arguments, `/auth` lists all available services:
	324	+
	325	+```
	326	+> /auth
	327	+Usage: /auth SERVICE
	328	+Available: dropbox, github, google, microsoft, notion, zoom
	329	+```
	330	+
	331	+With a service name, it runs the same auth chain as the CLI command:
	332	+
	333	+```
	334	+> /auth github
	335	+Github authentication successful (api_key).
	336	+```
	337	+
	338	+## Environment variables reference
	339	+
	340	+The following table summarizes all environment variables used by the authentication system:
	341	+
	342	+\| Service \| OAuth Client ID \| OAuth Client Secret \| API Key / Token \| Account ID \|
	343	+\|---------\|----------------\|--------------------\|--------------------\|------------\|
	344	+\| Google \| `GOOGLE_CLIENT_ID` \| `GOOGLE_CLIENT_SECRET` \| `GOOGLE_API_KEY` \| -- \|
	345	+\| Dropbox \| `DROPBOX_APP_KEY` \| `DROPBOX_APP_SECRET` \| `DROPBOX_ACCESS_TOKEN` \| -- \|
	346	+\| Zoom \| `ZOOM_CLIENT_ID` \| `ZOOM_CLIENT_SECRET` \| -- \| `ZOOM_ACCOUNT_ID` \|
	347	+\| Notion \| `NOTION_CLIENT_ID` \| `NOTION_CLIENT_SECRET` \| `NOTION_API_KEY` \| -- \|
	348	+\| GitHub \| `GITHUB_CLIENT_ID` \| `GITHUB_CLIENT_SECRET` \| `GITHUB_TOKEN` \| -- \|
	349	+\| Microsoft \| `MICROSOFT_CLIENT_ID` \| `MICROSOFT_CLIENT_SECRET` \| -- \| -- \|
	350	+
	351	+## Python API
	352	+
	353	+### AuthConfig
	354	+
	355	+The `AuthConfig` dataclass defines the authentication configuration for a service. It holds OAuth endpoints, credential references, scopes, and token storage paths.
	356	+
	357	+```python
	358	+from video_processor.auth import AuthConfig
	359	+
	360	+config = AuthConfig(
	361	+ service="myservice",
	362	+ oauth_authorize_url="https://example.com/oauth/authorize",
	363	+ oauth_token_url="https://example.com/oauth/token",
	364	+ client_id_env="MYSERVICE_CLIENT_ID",
	365	+ client_secret_env="MYSERVICE_CLIENT_SECRET",
	366	+ api_key_env="MYSERVICE_API_KEY",
	367	+ scopes=["read", "write"],
	368	+)
	369	+```
	370	+
	371	+Key fields:
	372	+
	373	+\| Field \| Type \| Description \|
	374	+\|-------\|------\|-------------\|
	375	+\| `service` \| `str` \| Service identifier (used for token filename) \|
	376	+\| `oauth_authorize_url` \| `Optional[str]` \| OAuth authorization endpoint \|
	377	+\| `oauth_token_url` \| `Optional[str]` \| OAuth token endpoint \|
	378	+\| `client_id` / `client_id_env` \| `Optional[str]` \| Client ID value or env var name \|
	379	+\| `client_secret` / `client_secret_env` \| `Optional[str]` \| Client secret value or env var name \|
	380	+\| `api_key_env` \| `Optional[str]` \| Environment variable for API key fallback \|
	381	+\| `scopes` \| `List[str]` \| OAuth scopes to request \|
	382	+\| `redirect_uri` \| `str` \| Redirect URI (default: `urn:ietf:wg:oauth:2.0:oob`) \|
	383	+\| `account_id` / `account_id_env` \| `Optional[str]` \| Account ID for client credentials grant \|
	384	+\| `token_path` \| `Optional[Path]` \| Override token storage path \|
	385	+
	386	+Resolved properties:
	387	+
	388	+- `resolved_client_id` -- Returns the client ID from the direct value or environment variable.
	389	+- `resolved_client_secret` -- Returns the client secret from the direct value or environment variable.
	390	+- `resolved_api_key` -- Returns the API key from the environment variable.
	391	+- `resolved_account_id` -- Returns the account ID from the direct value or environment variable.
	392	+- `resolved_token_path` -- Returns the token file path (default: `~/.planopticon/{service}_token.json`).
	393	+- `supports_oauth` -- Returns `True` if both OAuth endpoints are configured.
	394	+
	395	+### OAuthManager
	396	+
	397	+The `OAuthManager` class manages the full authentication lifecycle for a service.
	398	+
	399	+```python
	400	+from video_processor.auth import OAuthManager, AuthConfig
	401	+
	402	+config = AuthConfig(
	403	+ service="notion",
	404	+ oauth_authorize_url="https://api.notion.com/v1/oauth/authorize",
	405	+ oauth_token_url="https://api.notion.com/v1/oauth/token",
	406	+ client_id_env="NOTION_CLIENT_ID",
	407	+ client_secret_env="NOTION_CLIENT_SECRET",
	408	+ api_key_env="NOTION_API_KEY",
	409	+ scopes=["read_content"],
	410	+)
	411	+manager = OAuthManager(config)
	412	+
	413	+# Full auth chain -- returns AuthResult
	414	+result = manager.authenticate()
	415	+if result.success:
	416	+ print(f"Authenticated via {result.method}")
	417	+ print(f"Token: {result.access_token[:20]}...")
	418	+
	419	+# Convenience method -- returns just the token string or None
	420	+token = manager.get_token()
	421	+
	422	+# Clear saved token (logout)
	423	+manager.clear_token()
	424	+```
	425	+
	426	+AuthResult fields:
	427	+
	428	+\| Field \| Type \| Description \|
	429	+\|-------\|------\|-------------\|
	430	+\| `success` \| `bool` \| Whether authentication succeeded \|
	431	+\| `access_token` \| `Optional[str]` \| The access token (if successful) \|
	432	+\| `method` \| `Optional[str]` \| One of: `saved_token`, `oauth_pkce`, `client_credentials`, `api_key` \|
	433	+\| `expires_at` \| `Optional[float]` \| Token expiry as a Unix timestamp \|
	434	+\| `refresh_token` \| `Optional[str]` \| Refresh token (if provided) \|
	435	+\| `error` \| `Optional[str]` \| Error message (if unsuccessful) \|
	436	+
	437	+### Pre-built configs
	438	+
	439	+PlanOpticon ships with pre-built `AuthConfig` instances for all six supported services. Access them via convenience functions:
	440	+
	441	+```python
	442	+from video_processor.auth import get_auth_config, get_auth_manager
	443	+
	444	+# Get just the config
	445	+config = get_auth_config("zoom")
	446	+
	447	+# Get a ready-to-use manager
	448	+manager = get_auth_manager("github")
	449	+token = manager.get_token()
	450	+```
	451	+
	452	+### Building custom connectors
	453	+
	454	+To add authentication for a new service, create an `AuthConfig` with the service's OAuth endpoints and credential environment variables:
	455	+
	456	+```python
	457	+from video_processor.auth import AuthConfig, OAuthManager
	458	+
	459	+config = AuthConfig(
	460	+ service="slack",
	461	+ oauth_authorize_url="https://slack.com/oauth/v2/authorize",
	462	+ oauth_token_url="https://slack.com/api/oauth.v2.access",
	463	+ client_id_env="SLACK_CLIENT_ID",
	464	+ client_secret_env="SLACK_CLIENT_SECRET",
	465	+ api_key_env="SLACK_BOT_TOKEN",
	466	+ scopes=["channels:read", "channels:history"],
	467	+)
	468	+
	469	+manager = OAuthManager(config)
	470	+result = manager.authenticate()
	471	+```
	472	+
	473	+The token will be saved to `~/.planopticon/slack_token.json` and automatically refreshed on subsequent calls.
	474	+
	475	+## Troubleshooting
	476	+
	477	+### "No auth method available for {service}"
	478	+
	479	+This means none of the four auth methods succeeded. Check that:
	480	+
	481	+- The required environment variables are set and non-empty.
	482	+- For OAuth: both the client ID and client secret (or app key/secret) are set.
	483	+- For API key fallback: the correct environment variable is set.
	484	+
	485	+The error message includes hints about which variables to set.
	486	+
	487	+### Token refresh fails
	488	+
	489	+If automatic token refresh fails, PlanOpticon falls back to the next auth method in the chain. Common causes:
	490	+
	491	+- The refresh token has been revoked (e.g., you changed your password or revoked app access).
	492	+- The OAuth app's client secret has changed.
	493	+- The service requires re-authorization after a certain period.
	494	+
	495	+To resolve, clear the token and re-authenticate:
	496	+
	497	+```bash
	498	+planopticon auth google --logout
	499	+planopticon auth google
	500	+```
	501	+
	502	+### OAuth PKCE flow does not open a browser
	503	+
	504	+If the browser does not open automatically, PlanOpticon prints the authorization URL to the terminal. Copy and paste it into your browser manually. After authorizing, paste the authorization code back into the terminal prompt.
	505	+
	506	+### "requests not installed"
	507	+
	508	+The OAuth flows require the `requests` library. It is included as a dependency of PlanOpticon, but if you installed PlanOpticon in a minimal environment, install it manually:
	509	+
	510	+```bash
	511	+pip install requests
	512	+```
	513	+
	514	+### Permission denied on token file
	515	+
	516	+PlanOpticon needs write access to `~/.planopticon/`. If the directory or token files have restrictive permissions, adjust them:
	517	+
	518	+```bash
	519	+chmod 700 ~/.planopticon
	520	+chmod 600 ~/.planopticon/*_token.json
	521	+```
	522	+
	523	+### Microsoft authentication uses the `/common` tenant
	524	+
	525	+The default Microsoft OAuth configuration uses the `common` tenant endpoint (`login.microsoftonline.com/common/...`), which supports both personal Microsoft accounts and Azure AD organizational accounts. If your organization requires a specific tenant, you can create a custom `AuthConfig` with the tenant-specific URLs.

	--- a/docs/guide/authentication.md
	+++ b/docs/guide/authentication.md
	@@ -0,0 +1,525 @@

	--- a/docs/guide/authentication.md
	+++ b/docs/guide/authentication.md
	@@ -0,0 +1,525 @@
1	# Authentication
2
3	PlanOpticon uses a unified authentication system to connect with cloud services for fetching recordings, documents, and other content. The system is OAuth-first: it prefers OAuth 2.0 flows for security and token management, but falls back to API keys when OAuth is not configured.
4
5	## Auth strategy overview
6
7	PlanOpticon supports six cloud services out of the box: Google, Dropbox, Zoom, Notion, GitHub, and Microsoft. Each service uses the same authentication chain, implemented through the `OAuthManager` class. You configure credentials once (via environment variables or directly), and PlanOpticon handles token acquisition, storage, refresh, and fallback automatically.
8
9	All authentication state is managed through the `planopticon auth` CLI command, the `/auth` companion REPL command, or programmatically via the Python API.
10
11	## The auth chain
12
13	When you authenticate with a service, PlanOpticon tries the following methods in order. It stops at the first one that succeeds:
14
15	1. Saved token -- Checks `~/.planopticon/{service}_token.json` for a previously saved token. If the token has not expired, it is used immediately. If it has expired but a refresh token is available, PlanOpticon attempts an automatic token refresh.
16
17	2. Client Credentials grant (Server-to-Server) -- If an `account_id` is configured (e.g., `ZOOM_ACCOUNT_ID`), PlanOpticon attempts a client credentials grant. This is a non-interactive flow suitable for automated pipelines and server-side integrations. No browser is required.
18
19	3. OAuth 2.0 Authorization Code with PKCE (interactive) -- If a client ID is configured and OAuth endpoints are available, PlanOpticon initiates an interactive OAuth PKCE flow. It opens a browser to the service's authorization page, waits for you to paste the authorization code, and exchanges it for tokens. The tokens are saved for future use.
20
21	4. API key fallback -- If no OAuth method succeeds, PlanOpticon checks for a service-specific API key environment variable (e.g., `GITHUB_TOKEN`, `NOTION_API_KEY`). This is the simplest setup but may have reduced capabilities compared to OAuth.
22
23	If none of the four methods succeed, PlanOpticon returns an error with hints about which environment variables to set.
24
25	## Token storage
26
27	Tokens are persisted as JSON files in `~/.planopticon/`:
28
29	```
30	~/.planopticon/
31	google_token.json
32	dropbox_token.json
33	zoom_token.json
34	notion_token.json
35	github_token.json
36	microsoft_token.json
37	```
38
39	Each token file contains:
40
41	\| Field \| Description \|
42	\|-------\|-------------\|
43	\| `access_token` \| The current access token \|
44	\| `refresh_token` \| Refresh token for automatic renewal (if provided by the service) \|
45	\| `expires_at` \| Unix timestamp when the token expires (with a 60-second safety margin) \|
46	\| `client_id` \| The client ID used for this token (for refresh) \|
47	\| `client_secret` \| The client secret used (for refresh) \|
48
49	The `~/.planopticon/` directory is created automatically on first use. Token files are overwritten on each successful authentication or refresh.
50
51	To remove a saved token, use `planopticon auth <service> --logout` or delete the file directly.
52
53	## Supported services
54
55	### Google
56
57	Google authentication provides access to Google Drive and Google Docs for fetching documents, recordings, and other content.
58
59	Scopes requested:
60
61	- `https://www.googleapis.com/auth/drive.readonly`
62	- `https://www.googleapis.com/auth/documents.readonly`
63
64	Environment variables:
65
66	\| Variable \| Required \| Description \|
67	\|----------\|----------\|-------------\|
68	\| `GOOGLE_CLIENT_ID` \| For OAuth \| OAuth 2.0 Client ID from Google Cloud Console \|
69	\| `GOOGLE_CLIENT_SECRET` \| For OAuth \| OAuth 2.0 Client Secret \|
70	\| `GOOGLE_API_KEY` \| Fallback \| API key (limited access, no user-specific data) \|
71
72	OAuth app setup:
73
74	1. Go to the [Google Cloud Console](https://console.cloud.google.com/).
75	2. Create a project (or select an existing one).
76	3. Navigate to APIs & Services > Credentials.
77	4. Click Create Credentials > OAuth client ID.
78	5. Choose Desktop app as the application type.
79	6. Copy the Client ID and Client Secret.
80	7. Under APIs & Services > Library, enable the Google Drive API and Google Docs API.
81	8. Set the environment variables:
82
83	```bash
84	export GOOGLE_CLIENT_ID="your-client-id.apps.googleusercontent.com"
85	export GOOGLE_CLIENT_SECRET="your-client-secret"
86	```
87
88	Service account fallback: For automated pipelines, you can use a Google service account instead of OAuth. Generate a service account key JSON file from the Google Cloud Console and set `GOOGLE_APPLICATION_CREDENTIALS` to point to it. The PlanOpticon Google Workspace connector (`planopticon gws`) uses the `gws` CLI which has its own auth flow via `gws auth login`.
89
90	### Dropbox
91
92	Dropbox authentication provides access to files stored in Dropbox.
93
94	Environment variables:
95
96	\| Variable \| Required \| Description \|
97	\|----------\|----------\|-------------\|
98	\| `DROPBOX_APP_KEY` \| For OAuth \| App key from the Dropbox App Console \|
99	\| `DROPBOX_APP_SECRET` \| For OAuth \| App secret \|
100	\| `DROPBOX_ACCESS_TOKEN` \| Fallback \| Long-lived access token (for quick setup) \|
101
102	OAuth app setup:
103
104	1. Go to the [Dropbox App Console](https://www.dropbox.com/developers/apps).
105	2. Click Create App.
106	3. Choose Scoped access and Full Dropbox (or App folder for restricted access).
107	4. Copy the App key and App secret from the Settings tab.
108	5. Set the environment variables:
109
110	```bash
111	export DROPBOX_APP_KEY="your-app-key"
112	export DROPBOX_APP_SECRET="your-app-secret"
113	```
114
115	Access token shortcut: For quick testing, you can generate an access token directly from the app's Settings page in the Dropbox App Console and set it as `DROPBOX_ACCESS_TOKEN`. This bypasses OAuth entirely but the token may have a limited lifetime.
116
117	### Zoom
118
119	Zoom authentication provides access to cloud recordings, meeting metadata, and transcripts.
120
121	Environment variables:
122
123	\| Variable \| Required \| Description \|
124	\|----------\|----------\|-------------\|
125	\| `ZOOM_CLIENT_ID` \| For OAuth \| OAuth client ID from the Zoom Marketplace \|
126	\| `ZOOM_CLIENT_SECRET` \| For OAuth \| OAuth client secret \|
127	\| `ZOOM_ACCOUNT_ID` \| For S2S \| Account ID for Server-to-Server OAuth \|
128
129	Server-to-Server (recommended for automation):
130
131	When `ZOOM_ACCOUNT_ID` is set alongside `ZOOM_CLIENT_ID` and `ZOOM_CLIENT_SECRET`, PlanOpticon uses the client credentials grant (Server-to-Server OAuth). This is non-interactive and ideal for CI/CD pipelines and scheduled jobs.
132
133	1. Go to the [Zoom Marketplace](https://marketplace.zoom.us/).
134	2. Click Develop > Build App.
135	3. Choose Server-to-Server OAuth.
136	4. Copy the Account ID, Client ID, and Client Secret.
137	5. Add the required scopes: `recording:read:admin` (or `recording:read`).
138	6. Set the environment variables:
139
140	```bash
141	export ZOOM_CLIENT_ID="your-client-id"
142	export ZOOM_CLIENT_SECRET="your-client-secret"
143	export ZOOM_ACCOUNT_ID="your-account-id"
144	```
145
146	User-level OAuth PKCE:
147
148	If `ZOOM_ACCOUNT_ID` is not set, PlanOpticon falls back to the interactive OAuth PKCE flow. This opens a browser window for the user to authorize access.
149
150	1. In the Zoom Marketplace, create a General App (or OAuth app).
151	2. Set the redirect URI to `urn:ietf:wg:oauth:2.0:oob` (out-of-band).
152	3. Copy the Client ID and Client Secret.
153
154	### Notion
155
156	Notion authentication provides access to pages, databases, and content in your Notion workspace.
157
158	Environment variables:
159
160	\| Variable \| Required \| Description \|
161	\|----------\|----------\|-------------\|
162	\| `NOTION_CLIENT_ID` \| For OAuth \| OAuth client ID from the Notion Integrations page \|
163	\| `NOTION_CLIENT_SECRET` \| For OAuth \| OAuth client secret \|
164	\| `NOTION_API_KEY` \| Fallback \| Internal integration token \|
165
166	OAuth app setup:
167
168	1. Go to [My Integrations](https://www.notion.so/my-integrations) in Notion.
169	2. Click New integration.
170	3. Select Public integration (required for OAuth).
171	4. Copy the OAuth Client ID and Client Secret.
172	5. Set the redirect URI.
173	6. Set the environment variables:
174
175	```bash
176	export NOTION_CLIENT_ID="your-client-id"
177	export NOTION_CLIENT_SECRET="your-client-secret"
178	```
179
180	Internal integration (API key fallback):
181
182	For simpler setups, create an Internal integration from the Notion Integrations page. Copy the integration token and set it as `NOTION_API_KEY`. You must also share the relevant Notion pages/databases with the integration.
183
184	```bash
185	export NOTION_API_KEY="ntn_your-integration-token"
186	```
187
188	### GitHub
189
190	GitHub authentication provides access to repositories, issues, and organization data.
191
192	Scopes requested:
193
194	- `repo`
195	- `read:org`
196
197	Environment variables:
198
199	\| Variable \| Required \| Description \|
200	\|----------\|----------\|-------------\|
201	\| `GITHUB_CLIENT_ID` \| For OAuth \| OAuth App client ID \|
202	\| `GITHUB_CLIENT_SECRET` \| For OAuth \| OAuth App client secret \|
203	\| `GITHUB_TOKEN` \| Fallback \| Personal access token (classic or fine-grained) \|
204
205	OAuth app setup:
206
207	1. Go to GitHub > Settings > Developer Settings > OAuth Apps.
208	2. Click New OAuth App.
209	3. Set the Authorization callback URL to `urn:ietf:wg:oauth:2.0:oob`.
210	4. Copy the Client ID and generate a Client Secret.
211	5. Set the environment variables:
212
213	```bash
214	export GITHUB_CLIENT_ID="your-client-id"
215	export GITHUB_CLIENT_SECRET="your-client-secret"
216	```
217
218	Personal access token (recommended for most users):
219
220	The simplest approach is to create a Personal Access Token:
221
222	1. Go to GitHub > Settings > Developer Settings > Personal Access Tokens.
223	2. Generate a token with `repo` and `read:org` scopes.
224	3. Set it as `GITHUB_TOKEN`:
225
226	```bash
227	export GITHUB_TOKEN="ghp_your-token"
228	```
229
230	### Microsoft
231
232	Microsoft authentication provides access to Microsoft 365 resources via the Microsoft Graph API, including OneDrive, SharePoint, and Teams recordings.
233
234	Scopes requested:
235
236	- `https://graph.microsoft.com/OnlineMeetings.Read`
237	- `https://graph.microsoft.com/Files.Read`
238
239	Environment variables:
240
241	\| Variable \| Required \| Description \|
242	\|----------\|----------\|-------------\|
243	\| `MICROSOFT_CLIENT_ID` \| For OAuth \| Application (client) ID from Azure AD \|
244	\| `MICROSOFT_CLIENT_SECRET` \| For OAuth \| Client secret from Azure AD \|
245
246	Azure AD app registration:
247
248	1. Go to the [Azure Portal](https://portal.azure.com/).
249	2. Navigate to Azure Active Directory > App registrations.
250	3. Click New registration.
251	4. Name the application (e.g., "PlanOpticon").
252	5. Under Supported account types, select the appropriate option for your organization.
253	6. Set the redirect URI to `urn:ietf:wg:oauth:2.0:oob` with platform Mobile and desktop applications.
254	7. After registration, go to Certificates & secrets and create a new client secret.
255	8. Under API permissions, add:
256	- `OnlineMeetings.Read`
257	- `Files.Read`
258	9. Grant admin consent if required by your organization.
259	10. Set the environment variables:
260
261	```bash
262	export MICROSOFT_CLIENT_ID="your-application-id"
263	export MICROSOFT_CLIENT_SECRET="your-client-secret"
264	```
265
266	Microsoft 365 CLI: The `planopticon m365` commands use the `@pnp/cli-microsoft365` npm package, which has its own authentication flow via `m365 login`. This is separate from the OAuth flow described above.
267
268	## CLI usage
269
270	### `planopticon auth`
271
272	Authenticate with a cloud service or manage saved tokens.
273
274	```
275	planopticon auth SERVICE [--logout]
276	```
277
278	Arguments:
279
280	\| Argument \| Description \|
281	\|----------\|-------------\|
282	\| `SERVICE` \| One of: `google`, `dropbox`, `zoom`, `notion`, `github`, `microsoft` \|
283
284	Options:
285
286	\| Option \| Description \|
287	\|--------\|-------------\|
288	\| `--logout` \| Clear the saved token for the specified service \|
289
290	Examples:
291
292	```bash
293	# Authenticate with Google (triggers OAuth flow or uses saved token)
294	planopticon auth google
295
296	# Authenticate with Zoom
297	planopticon auth zoom
298
299	# Clear saved GitHub token
300	planopticon auth github --logout
301	```
302
303	On success, the command prints the authentication method used:
304
305	```
306	Google authentication successful (oauth_pkce).
307	```
308
309	or
310
311	```
312	Github authentication successful (api_key).
313	```
314
315	### Companion REPL `/auth`
316
317	Inside the interactive companion REPL (`planopticon -C` or `planopticon -I`), you can authenticate with services using the `/auth` command:
318
319	```
320	/auth SERVICE
321	```
322
323	Without arguments, `/auth` lists all available services:
324
325	```
326	> /auth
327	Usage: /auth SERVICE
328	Available: dropbox, github, google, microsoft, notion, zoom
329	```
330
331	With a service name, it runs the same auth chain as the CLI command:
332
333	```
334	> /auth github
335	Github authentication successful (api_key).
336	```
337
338	## Environment variables reference
339
340	The following table summarizes all environment variables used by the authentication system:
341
342	\| Service \| OAuth Client ID \| OAuth Client Secret \| API Key / Token \| Account ID \|
343	\|---------\|----------------\|--------------------\|--------------------\|------------\|
344	\| Google \| `GOOGLE_CLIENT_ID` \| `GOOGLE_CLIENT_SECRET` \| `GOOGLE_API_KEY` \| -- \|
345	\| Dropbox \| `DROPBOX_APP_KEY` \| `DROPBOX_APP_SECRET` \| `DROPBOX_ACCESS_TOKEN` \| -- \|
346	\| Zoom \| `ZOOM_CLIENT_ID` \| `ZOOM_CLIENT_SECRET` \| -- \| `ZOOM_ACCOUNT_ID` \|
347	\| Notion \| `NOTION_CLIENT_ID` \| `NOTION_CLIENT_SECRET` \| `NOTION_API_KEY` \| -- \|
348	\| GitHub \| `GITHUB_CLIENT_ID` \| `GITHUB_CLIENT_SECRET` \| `GITHUB_TOKEN` \| -- \|
349	\| Microsoft \| `MICROSOFT_CLIENT_ID` \| `MICROSOFT_CLIENT_SECRET` \| -- \| -- \|
350
351	## Python API
352
353	### AuthConfig
354
355	The `AuthConfig` dataclass defines the authentication configuration for a service. It holds OAuth endpoints, credential references, scopes, and token storage paths.
356
357	```python
358	from video_processor.auth import AuthConfig
359
360	config = AuthConfig(
361	service="myservice",
362	oauth_authorize_url="https://example.com/oauth/authorize",
363	oauth_token_url="https://example.com/oauth/token",
364	client_id_env="MYSERVICE_CLIENT_ID",
365	client_secret_env="MYSERVICE_CLIENT_SECRET",
366	api_key_env="MYSERVICE_API_KEY",
367	scopes=["read", "write"],
368	)
369	```
370
371	Key fields:
372
373	\| Field \| Type \| Description \|
374	\|-------\|------\|-------------\|
375	\| `service` \| `str` \| Service identifier (used for token filename) \|
376	\| `oauth_authorize_url` \| `Optional[str]` \| OAuth authorization endpoint \|
377	\| `oauth_token_url` \| `Optional[str]` \| OAuth token endpoint \|
378	\| `client_id` / `client_id_env` \| `Optional[str]` \| Client ID value or env var name \|
379	\| `client_secret` / `client_secret_env` \| `Optional[str]` \| Client secret value or env var name \|
380	\| `api_key_env` \| `Optional[str]` \| Environment variable for API key fallback \|
381	\| `scopes` \| `List[str]` \| OAuth scopes to request \|
382	\| `redirect_uri` \| `str` \| Redirect URI (default: `urn:ietf:wg:oauth:2.0:oob`) \|
383	\| `account_id` / `account_id_env` \| `Optional[str]` \| Account ID for client credentials grant \|
384	\| `token_path` \| `Optional[Path]` \| Override token storage path \|
385
386	Resolved properties:
387
388	- `resolved_client_id` -- Returns the client ID from the direct value or environment variable.
389	- `resolved_client_secret` -- Returns the client secret from the direct value or environment variable.
390	- `resolved_api_key` -- Returns the API key from the environment variable.
391	- `resolved_account_id` -- Returns the account ID from the direct value or environment variable.
392	- `resolved_token_path` -- Returns the token file path (default: `~/.planopticon/{service}_token.json`).
393	- `supports_oauth` -- Returns `True` if both OAuth endpoints are configured.
394
395	### OAuthManager
396
397	The `OAuthManager` class manages the full authentication lifecycle for a service.
398
399	```python
400	from video_processor.auth import OAuthManager, AuthConfig
401
402	config = AuthConfig(
403	service="notion",
404	oauth_authorize_url="https://api.notion.com/v1/oauth/authorize",
405	oauth_token_url="https://api.notion.com/v1/oauth/token",
406	client_id_env="NOTION_CLIENT_ID",
407	client_secret_env="NOTION_CLIENT_SECRET",
408	api_key_env="NOTION_API_KEY",
409	scopes=["read_content"],
410	)
411	manager = OAuthManager(config)
412
413	# Full auth chain -- returns AuthResult
414	result = manager.authenticate()
415	if result.success:
416	print(f"Authenticated via {result.method}")
417	print(f"Token: {result.access_token[:20]}...")
418
419	# Convenience method -- returns just the token string or None
420	token = manager.get_token()
421
422	# Clear saved token (logout)
423	manager.clear_token()
424	```
425
426	AuthResult fields:
427
428	\| Field \| Type \| Description \|
429	\|-------\|------\|-------------\|
430	\| `success` \| `bool` \| Whether authentication succeeded \|
431	\| `access_token` \| `Optional[str]` \| The access token (if successful) \|
432	\| `method` \| `Optional[str]` \| One of: `saved_token`, `oauth_pkce`, `client_credentials`, `api_key` \|
433	\| `expires_at` \| `Optional[float]` \| Token expiry as a Unix timestamp \|
434	\| `refresh_token` \| `Optional[str]` \| Refresh token (if provided) \|
435	\| `error` \| `Optional[str]` \| Error message (if unsuccessful) \|
436
437	### Pre-built configs
438
439	PlanOpticon ships with pre-built `AuthConfig` instances for all six supported services. Access them via convenience functions:
440
441	```python
442	from video_processor.auth import get_auth_config, get_auth_manager
443
444	# Get just the config
445	config = get_auth_config("zoom")
446
447	# Get a ready-to-use manager
448	manager = get_auth_manager("github")
449	token = manager.get_token()
450	```
451
452	### Building custom connectors
453
454	To add authentication for a new service, create an `AuthConfig` with the service's OAuth endpoints and credential environment variables:
455
456	```python
457	from video_processor.auth import AuthConfig, OAuthManager
458
459	config = AuthConfig(
460	service="slack",
461	oauth_authorize_url="https://slack.com/oauth/v2/authorize",
462	oauth_token_url="https://slack.com/api/oauth.v2.access",
463	client_id_env="SLACK_CLIENT_ID",
464	client_secret_env="SLACK_CLIENT_SECRET",
465	api_key_env="SLACK_BOT_TOKEN",
466	scopes=["channels:read", "channels:history"],
467	)
468
469	manager = OAuthManager(config)
470	result = manager.authenticate()
471	```
472
473	The token will be saved to `~/.planopticon/slack_token.json` and automatically refreshed on subsequent calls.
474
475	## Troubleshooting
476
477	### "No auth method available for {service}"
478
479	This means none of the four auth methods succeeded. Check that:
480
481	- The required environment variables are set and non-empty.
482	- For OAuth: both the client ID and client secret (or app key/secret) are set.
483	- For API key fallback: the correct environment variable is set.
484
485	The error message includes hints about which variables to set.
486
487	### Token refresh fails
488
489	If automatic token refresh fails, PlanOpticon falls back to the next auth method in the chain. Common causes:
490
491	- The refresh token has been revoked (e.g., you changed your password or revoked app access).
492	- The OAuth app's client secret has changed.
493	- The service requires re-authorization after a certain period.
494
495	To resolve, clear the token and re-authenticate:
496
497	```bash
498	planopticon auth google --logout
499	planopticon auth google
500	```
501
502	### OAuth PKCE flow does not open a browser
503
504	If the browser does not open automatically, PlanOpticon prints the authorization URL to the terminal. Copy and paste it into your browser manually. After authorizing, paste the authorization code back into the terminal prompt.
505
506	### "requests not installed"
507
508	The OAuth flows require the `requests` library. It is included as a dependency of PlanOpticon, but if you installed PlanOpticon in a minimal environment, install it manually:
509
510	```bash
511	pip install requests
512	```
513
514	### Permission denied on token file
515
516	PlanOpticon needs write access to `~/.planopticon/`. If the directory or token files have restrictive permissions, adjust them:
517
518	```bash
519	chmod 700 ~/.planopticon
520	chmod 600 ~/.planopticon/*_token.json
521	```
522
523	### Microsoft authentication uses the `/common` tenant
524
525	The default Microsoft OAuth configuration uses the `common` tenant endpoint (`login.microsoftonline.com/common/...`), which supports both personal Microsoft accounts and Azure AD organizational accounts. If your organization requires a specific tenant, you can create a custom `AuthConfig` with the tenant-specific URLs.

M docs/guide/batch.md

+120 -10

		--- docs/guide/batch.md
		+++ docs/guide/batch.md
		@@ -10,11 +10,11 @@
10	10
11	11	Batch mode:
12	12
13	13	1. Scans the input directory for video files matching the pattern
14	14	2. Processes each video through the full single-video pipeline
15		-3. Merges knowledge graphs across all videos (case-insensitive entity dedup)
	15	+3. Merges knowledge graphs across all videos with fuzzy matching and conflict resolution
16	16	4. Generates a batch summary with aggregated stats and action items
17	17	5. Writes a batch manifest linking to per-video results
18	18
19	19	## File patterns
20	20
		@@ -30,32 +30,58 @@
30	30
31	31	```
32	32	output/
33	33	├── batch_manifest.json # Batch-level manifest
34	34	├── batch_summary.md # Aggregated summary
35		-├── knowledge_graph.json # Merged KG across all videos
	35	+├── knowledge_graph.db # Merged KG across all videos (SQLite, primary)
	36	+├── knowledge_graph.json # Merged KG across all videos (JSON export)
36	37	└── videos/
37	38	├── meeting-01/
38	39	│ ├── manifest.json
39	40	│ ├── transcript/
40	41	│ ├── diagrams/
	42	+ │ ├── captures/
41	43	│ └── results/
	44	+ │ ├── analysis.md
	45	+ │ ├── analysis.html
	46	+ │ ├── knowledge_graph.db
	47	+ │ ├── knowledge_graph.json
	48	+ │ ├── key_points.json
	49	+ │ └── action_items.json
42	50	└── meeting-02/
43	51	├── manifest.json
44	52	└── ...
45	53	```
46	54
47	55	## Knowledge graph merging
48	56
49		-When the same entity appears across multiple videos, PlanOpticon merges them:
50		-
51		-- Case-insensitive name matching
52		-- Descriptions are unioned
53		-- Occurrences are concatenated with source tracking
54		-- Relationships are deduplicated
55		-
56		-The merged knowledge graph is saved at the batch root and included in the batch summary as a mermaid diagram.
	57	+When the same entity appears across multiple videos, PlanOpticon merges them using a multi-strategy approach:
	58	+
	59	+### Entity deduplication
	60	+
	61	+- Case-insensitive exact matching -- `"kubernetes"` and `"Kubernetes"` are recognized as the same entity
	62	+- Fuzzy name matching -- Uses `SequenceMatcher` with a threshold of 0.85 to unify near-duplicate entities (e.g., `"K8s"` and `"k8s cluster"` may be matched depending on context)
	63	+- Descriptions are unioned -- All unique descriptions from each video are combined
	64	+- Occurrences are concatenated with source tracking -- Each occurrence retains its source video reference
	65	+
	66	+### Relationship deduplication
	67	+
	68	+- Relationships are deduplicated by (source, target, type) tuple
	69	+- Descriptions from duplicate relationships are merged
	70	+
	71	+### Type conflict resolution
	72	+
	73	+When the same entity appears with different types across videos, PlanOpticon uses a specificity ranking to resolve the conflict. More specific types are preferred over general ones:
	74	+
	75	+- `technology` > `concept`
	76	+- `person` > `concept`
	77	+- `organization` > `concept`
	78	+- And so on through the full type hierarchy
	79	+
	80	+This ensures that an entity initially classified as a generic `concept` in one video gets upgraded to `technology` if it is identified more specifically in another.
	81	+
	82	+The merged knowledge graph is saved at the batch root in both SQLite (`knowledge_graph.db`) and JSON (`knowledge_graph.json`) formats, and is included in the batch summary as a Mermaid diagram.
57	83
58	84	## Error handling
59	85
60	86	If a video fails to process, the batch continues. Failed videos are recorded in the batch manifest with error details:
61	87
		@@ -64,5 +90,89 @@
64	90	"video_name": "corrupted-file",
65	91	"status": "failed",
66	92	"error": "Audio extraction failed: no audio track found"
67	93	}
68	94	```
	95	+
	96	+The batch manifest tracks completion status:
	97	+
	98	+```json
	99	+{
	100	+ "title": "Sprint Reviews",
	101	+ "total_videos": 5,
	102	+ "completed_videos": 4,
	103	+ "failed_videos": 1,
	104	+ "total_diagrams": 12,
	105	+ "total_action_items": 23,
	106	+ "total_key_points": 45,
	107	+ "videos": [...],
	108	+ "batch_summary_md": "batch_summary.md",
	109	+ "merged_knowledge_graph_json": "knowledge_graph.json",
	110	+ "merged_knowledge_graph_db": "knowledge_graph.db"
	111	+}
	112	+```
	113	+
	114	+## Using batch results
	115	+
	116	+### Query the merged knowledge graph
	117	+
	118	+After batch processing completes, the merged knowledge graph at the batch root contains entities and relationships from all successfully processed videos. You can query it just like a single-video knowledge graph:
	119	+
	120	+```bash
	121	+# Show stats for the merged graph
	122	+planopticon query --db output/knowledge_graph.db
	123	+
	124	+# List all people mentioned across all videos
	125	+planopticon query --db output/knowledge_graph.db "entities --type person"
	126	+
	127	+# See what connects to an entity across all videos
	128	+planopticon query --db output/knowledge_graph.db "neighbors Alice"
	129	+
	130	+# Ask natural language questions about the combined content
	131	+planopticon query --db output/knowledge_graph.db "What technologies were discussed across all meetings?"
	132	+
	133	+# Interactive REPL for exploration
	134	+planopticon query --db output/knowledge_graph.db -I
	135	+```
	136	+
	137	+### Export merged results
	138	+
	139	+All export commands work with the merged knowledge graph:
	140	+
	141	+```bash
	142	+# Generate documents from merged KG
	143	+planopticon export markdown output/knowledge_graph.db -o ./docs
	144	+
	145	+# Export as Obsidian vault
	146	+planopticon export obsidian output/knowledge_graph.db -o ./vault
	147	+
	148	+# Generate a project-wide exchange file
	149	+planopticon export exchange output/knowledge_graph.db --name "Sprint Reviews Q4"
	150	+
	151	+# Generate a GitHub wiki
	152	+planopticon wiki generate output/knowledge_graph.db -o ./wiki
	153	+```
	154	+
	155	+### Classify for planning
	156	+
	157	+Run taxonomy classification on the merged graph to categorize entities across all videos:
	158	+
	159	+```bash
	160	+planopticon kg classify output/knowledge_graph.db
	161	+```
	162	+
	163	+### Use with the planning agent
	164	+
	165	+The planning agent can consume the merged knowledge graph for cross-video analysis and planning:
	166	+
	167	+```bash
	168	+planopticon agent --db output/knowledge_graph.db
	169	+```
	170	+
	171	+### Incremental batch processing
	172	+
	173	+If you add new videos to the recordings directory, you can re-run the batch command. Videos that have already been processed (with output directories present) will be detected via checkpoint/resume within each video's pipeline, making incremental processing efficient.
	174	+
	175	+```bash
	176	+# Add new recordings to the folder, then re-run
	177	+planopticon batch -i ./recordings -o ./output --title "Sprint Reviews"
	178	+```
69	179
70	180	ADDED docs/guide/companion.md
71	181	ADDED docs/guide/document-ingestion.md
72	182	ADDED docs/guide/export.md
73	183	ADDED docs/guide/knowledge-graphs.md

	--- docs/guide/batch.md
	+++ docs/guide/batch.md
	@@ -10,11 +10,11 @@
10
11	Batch mode:
12
13	1. Scans the input directory for video files matching the pattern
14	2. Processes each video through the full single-video pipeline
15	3. Merges knowledge graphs across all videos (case-insensitive entity dedup)
16	4. Generates a batch summary with aggregated stats and action items
17	5. Writes a batch manifest linking to per-video results
18
19	## File patterns
20
	@@ -30,32 +30,58 @@
30
31	```
32	output/
33	├── batch_manifest.json # Batch-level manifest
34	├── batch_summary.md # Aggregated summary
35	├── knowledge_graph.json # Merged KG across all videos

36	└── videos/
37	├── meeting-01/
38	│ ├── manifest.json
39	│ ├── transcript/
40	│ ├── diagrams/

41	│ └── results/






42	└── meeting-02/
43	├── manifest.json
44	└── ...
45	```
46
47	## Knowledge graph merging
48
49	When the same entity appears across multiple videos, PlanOpticon merges them:
50
51	- Case-insensitive name matching
52	- Descriptions are unioned
53	- Occurrences are concatenated with source tracking
54	- Relationships are deduplicated
55
56	The merged knowledge graph is saved at the batch root and included in the batch summary as a mermaid diagram.


















57
58	## Error handling
59
60	If a video fails to process, the batch continues. Failed videos are recorded in the batch manifest with error details:
61
	@@ -64,5 +90,89 @@
64	"video_name": "corrupted-file",
65	"status": "failed",
66	"error": "Audio extraction failed: no audio track found"
67	}
68	```




















































































69
70	DDED docs/guide/companion.md
71	DDED docs/guide/document-ingestion.md
72	DDED docs/guide/export.md
73	DDED docs/guide/knowledge-graphs.md

	--- docs/guide/batch.md
	+++ docs/guide/batch.md
	@@ -10,11 +10,11 @@
10
11	Batch mode:
12
13	1. Scans the input directory for video files matching the pattern
14	2. Processes each video through the full single-video pipeline
15	3. Merges knowledge graphs across all videos with fuzzy matching and conflict resolution
16	4. Generates a batch summary with aggregated stats and action items
17	5. Writes a batch manifest linking to per-video results
18
19	## File patterns
20
	@@ -30,32 +30,58 @@
30
31	```
32	output/
33	├── batch_manifest.json # Batch-level manifest
34	├── batch_summary.md # Aggregated summary
35	├── knowledge_graph.db # Merged KG across all videos (SQLite, primary)
36	├── knowledge_graph.json # Merged KG across all videos (JSON export)
37	└── videos/
38	├── meeting-01/
39	│ ├── manifest.json
40	│ ├── transcript/
41	│ ├── diagrams/
42	│ ├── captures/
43	│ └── results/
44	│ ├── analysis.md
45	│ ├── analysis.html
46	│ ├── knowledge_graph.db
47	│ ├── knowledge_graph.json
48	│ ├── key_points.json
49	│ └── action_items.json
50	└── meeting-02/
51	├── manifest.json
52	└── ...
53	```
54
55	## Knowledge graph merging
56
57	When the same entity appears across multiple videos, PlanOpticon merges them using a multi-strategy approach:
58
59	### Entity deduplication
60
61	- Case-insensitive exact matching -- `"kubernetes"` and `"Kubernetes"` are recognized as the same entity
62	- Fuzzy name matching -- Uses `SequenceMatcher` with a threshold of 0.85 to unify near-duplicate entities (e.g., `"K8s"` and `"k8s cluster"` may be matched depending on context)
63	- Descriptions are unioned -- All unique descriptions from each video are combined
64	- Occurrences are concatenated with source tracking -- Each occurrence retains its source video reference
65
66	### Relationship deduplication
67
68	- Relationships are deduplicated by (source, target, type) tuple
69	- Descriptions from duplicate relationships are merged
70
71	### Type conflict resolution
72
73	When the same entity appears with different types across videos, PlanOpticon uses a specificity ranking to resolve the conflict. More specific types are preferred over general ones:
74
75	- `technology` > `concept`
76	- `person` > `concept`
77	- `organization` > `concept`
78	- And so on through the full type hierarchy
79
80	This ensures that an entity initially classified as a generic `concept` in one video gets upgraded to `technology` if it is identified more specifically in another.
81
82	The merged knowledge graph is saved at the batch root in both SQLite (`knowledge_graph.db`) and JSON (`knowledge_graph.json`) formats, and is included in the batch summary as a Mermaid diagram.
83
84	## Error handling
85
86	If a video fails to process, the batch continues. Failed videos are recorded in the batch manifest with error details:
87
	@@ -64,5 +90,89 @@
90	"video_name": "corrupted-file",
91	"status": "failed",
92	"error": "Audio extraction failed: no audio track found"
93	}
94	```
95
96	The batch manifest tracks completion status:
97
98	```json
99	{
100	"title": "Sprint Reviews",
101	"total_videos": 5,
102	"completed_videos": 4,
103	"failed_videos": 1,
104	"total_diagrams": 12,
105	"total_action_items": 23,
106	"total_key_points": 45,
107	"videos": [...],
108	"batch_summary_md": "batch_summary.md",
109	"merged_knowledge_graph_json": "knowledge_graph.json",
110	"merged_knowledge_graph_db": "knowledge_graph.db"
111	}
112	```
113
114	## Using batch results
115
116	### Query the merged knowledge graph
117
118	After batch processing completes, the merged knowledge graph at the batch root contains entities and relationships from all successfully processed videos. You can query it just like a single-video knowledge graph:
119
120	```bash
121	# Show stats for the merged graph
122	planopticon query --db output/knowledge_graph.db
123
124	# List all people mentioned across all videos
125	planopticon query --db output/knowledge_graph.db "entities --type person"
126
127	# See what connects to an entity across all videos
128	planopticon query --db output/knowledge_graph.db "neighbors Alice"
129
130	# Ask natural language questions about the combined content
131	planopticon query --db output/knowledge_graph.db "What technologies were discussed across all meetings?"
132
133	# Interactive REPL for exploration
134	planopticon query --db output/knowledge_graph.db -I
135	```
136
137	### Export merged results
138
139	All export commands work with the merged knowledge graph:
140
141	```bash
142	# Generate documents from merged KG
143	planopticon export markdown output/knowledge_graph.db -o ./docs
144
145	# Export as Obsidian vault
146	planopticon export obsidian output/knowledge_graph.db -o ./vault
147
148	# Generate a project-wide exchange file
149	planopticon export exchange output/knowledge_graph.db --name "Sprint Reviews Q4"
150
151	# Generate a GitHub wiki
152	planopticon wiki generate output/knowledge_graph.db -o ./wiki
153	```
154
155	### Classify for planning
156
157	Run taxonomy classification on the merged graph to categorize entities across all videos:
158
159	```bash
160	planopticon kg classify output/knowledge_graph.db
161	```
162
163	### Use with the planning agent
164
165	The planning agent can consume the merged knowledge graph for cross-video analysis and planning:
166
167	```bash
168	planopticon agent --db output/knowledge_graph.db
169	```
170
171	### Incremental batch processing
172
173	If you add new videos to the recordings directory, you can re-run the batch command. Videos that have already been processed (with output directories present) will be detected via checkpoint/resume within each video's pipeline, making incremental processing efficient.
174
175	```bash
176	# Add new recordings to the folder, then re-run
177	planopticon batch -i ./recordings -o ./output --title "Sprint Reviews"
178	```
179
180	DDED docs/guide/companion.md
181	DDED docs/guide/document-ingestion.md
182	DDED docs/guide/export.md
183	DDED docs/guide/knowledge-graphs.md

A docs/guide/companion.md

+531

		--- a/docs/guide/companion.md
		+++ b/docs/guide/companion.md
		@@ -0,0 +1,531 @@
	1	+# Interactive Companion REPL
	2	+
	3	+The PlanOpticon Companion is an interactive Read-Eval-Print Loop (REPL) that provides a conversational interface to PlanOpticon's full feature set. It combines workspace awareness, knowledge graph querying, LLM-powered chat, and planning agent skills into a single session.
	4	+
	5	+Use the Companion when you want to explore a knowledge graph interactively, ask natural-language questions about extracted content, generate planning artifacts on the fly, or switch between providers and models without restarting.
	6	+
	7	+---
	8	+
	9	+## Launching the Companion
	10	+
	11	+There are three equivalent ways to start the Companion.
	12	+
	13	+### As a subcommand
	14	+
	15	+```bash
	16	+planopticon companion
	17	+```
	18	+
	19	+### With the `--chat` / `-C` flag
	20	+
	21	+```bash
	22	+planopticon --chat
	23	+planopticon -C
	24	+```
	25	+
	26	+These flags launch the Companion directly from the top-level CLI, without invoking a subcommand.
	27	+
	28	+### With options
	29	+
	30	+The `companion` subcommand accepts options for specifying knowledge base paths, LLM provider, and model:
	31	+
	32	+```bash
	33	+# Point at a specific knowledge base
	34	+planopticon companion --kb ./results
	35	+
	36	+# Use a specific provider
	37	+planopticon companion -p anthropic
	38	+
	39	+# Use a specific model
	40	+planopticon companion --chat-model gpt-4o
	41	+
	42	+# Combine options
	43	+planopticon companion --kb ./results -p openai --chat-model gpt-4o
	44	+```
	45	+
	46	+\| Option \| Description \|
	47	+\|---\|---\|
	48	+\| `--kb PATH` \| Path to a knowledge graph file or directory (repeatable) \|
	49	+\| `-p, --provider NAME` \| LLM provider: `auto`, `openai`, `anthropic`, `gemini`, `ollama`, `azure`, `together`, `fireworks`, `cerebras`, `xai` \|
	50	+\| `--chat-model NAME` \| Override the default chat model for the selected provider \|
	51	+
	52	+---
	53	+
	54	+## Auto-discovery
	55	+
	56	+On startup, the Companion automatically scans the workspace for relevant files:
	57	+
	58	+Knowledge graphs. The Companion uses `find_nearest_graph()` to locate the closest `knowledge_graph.db` or `knowledge_graph.json` file. It searches the current directory, common output subdirectories (`results/`, `output/`, `knowledge-base/`), recursively downward (up to 4 levels), and upward through parent directories. SQLite `.db` files are preferred over `.json` files.
	59	+
	60	+Videos. The current directory is scanned for files with `.mp4`, `.mkv`, and `.webm` extensions.
	61	+
	62	+Documents. The current directory is scanned for files with `.md`, `.pdf`, and `.docx` extensions.
	63	+
	64	+LLM provider. If `--provider` is set to `auto` (the default), the Companion attempts to initialise a provider using any available API key in the environment (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, etc.).
	65	+
	66	+All discovered context is displayed in the welcome banner:
	67	+
	68	+```
	69	+ PlanOpticon Companion
	70	+ Interactive planning REPL
	71	+
	72	+ Knowledge graph: knowledge_graph.db (42 entities, 87 relationships)
	73	+ Videos: meeting-2024-01-15.mp4, sprint-review.mp4
	74	+ Docs: requirements.md, architecture.pdf
	75	+ LLM provider: openai (model: gpt-4o)
	76	+
	77	+ Type /help for commands, or ask a question.
	78	+```
	79	+
	80	+If no knowledge graph is found, the banner shows "No knowledge graph loaded." Commands that require a KG will return an appropriate message rather than failing silently.
	81	+
	82	+---
	83	+
	84	+## Slash Commands
	85	+
	86	+The Companion supports 18 slash commands. Type `/help` at the prompt to see the full list.
	87	+
	88	+### /help
	89	+
	90	+Display all available commands with brief descriptions.
	91	+
	92	+```
	93	+planopticon> /help
	94	+Available commands:
	95	+ /help Show this help
	96	+ /status Workspace status
	97	+ /skills List available skills
	98	+ /entities [--type T] List KG entities
	99	+ /search TERM Search entities by name
	100	+ /neighbors ENTITY Show entity relationships
	101	+ /export FORMAT Export KG (markdown, obsidian, notion, csv)
	102	+ /analyze PATH Analyze a video/doc
	103	+ /ingest PATH Ingest a file into the KG
	104	+ /auth SERVICE Authenticate with a cloud service
	105	+ /provider [NAME] List or switch LLM provider
	106	+ /model [NAME] Show or switch chat model
	107	+ /run SKILL Run a skill by name
	108	+ /plan Run project_plan skill
	109	+ /prd Run PRD skill
	110	+ /tasks Run task_breakdown skill
	111	+ /quit, /exit Exit companion
	112	+
	113	+Any other input is sent to the chat agent (requires LLM).
	114	+```
	115	+
	116	+### /status
	117	+
	118	+Show a summary of the current workspace state: loaded knowledge graph (with entity and relationship counts, broken down by entity type), number of discovered videos and documents, and whether an LLM provider is active.
	119	+
	120	+```
	121	+planopticon> /status
	122	+Workspace status:
	123	+ KG: /home/user/project/results/knowledge_graph.db (42 entities, 87 relationships)
	124	+ technology: 15
	125	+ person: 12
	126	+ concept: 10
	127	+ organization: 5
	128	+ Videos: 2 found
	129	+ Docs: 3 found
	130	+ Provider: active
	131	+```
	132	+
	133	+### /skills
	134	+
	135	+List all registered planning agent skills with their names and descriptions. These are the skills that can be invoked via `/run`.
	136	+
	137	+```
	138	+planopticon> /skills
	139	+Available skills:
	140	+ project_plan: Generate a structured project plan from knowledge graph
	141	+ prd: Generate a product requirements document (PRD) / feature spec
	142	+ roadmap: Generate a product/project roadmap
	143	+ task_breakdown: Break down goals into tasks with dependencies
	144	+ github_issues: Generate GitHub issues from task breakdown
	145	+ requirements_chat: Interactive requirements gathering via guided questions
	146	+ doc_generator: Generate technical documentation, ADRs, or meeting notes
	147	+ artifact_export: Export artifacts in agent-ready formats
	148	+ cli_adapter: Push artifacts to external tools via their CLIs
	149	+ notes_export: Export knowledge graph as structured notes (Obsidian, Notion)
	150	+ wiki_generator: Generate a GitHub wiki from knowledge graph and artifacts
	151	+```
	152	+
	153	+### /entities [--type TYPE]
	154	+
	155	+List entities from the loaded knowledge graph. Optionally filter by entity type.
	156	+
	157	+```
	158	+planopticon> /entities
	159	+Found 42 entities
	160	+ [technology] Python -- General-purpose programming language
	161	+ [person] Alice -- Lead engineer on the project
	162	+ [concept] Microservices -- Architectural pattern discussed
	163	+ ...
	164	+
	165	+planopticon> /entities --type person
	166	+Found 12 entities
	167	+ [person] Alice -- Lead engineer on the project
	168	+ [person] Bob -- Product manager
	169	+ ...
	170	+```
	171