PlanOpticon

planopticon / docs / guide / single-video.md
Source Rendered
Blame History Raw 245 lines
1
# Single Video Analysis
2
3
## Basic usage
4
5
```bash
6
planopticon analyze -i recording.mp4 -o ./output
7
```
8
9
## What happens
10
11
The pipeline runs these steps in order:
12
13
1. **Frame extraction** -- Samples frames using change detection for transitions plus periodic capture (every 30s) for slow-evolving content like document scrolling
14
2. **People frame filtering** -- OpenCV face detection automatically removes webcam/video conference frames, keeping only shared content (slides, documents, screen shares)
15
3. **Audio extraction** -- Extracts audio track to WAV
16
4. **Transcription** -- Sends audio to speech-to-text (Whisper or Gemini). If `--speakers` is provided, speaker diarization hints are passed to the provider.
17
5. **Diagram detection** -- Vision model classifies each frame as diagram/chart/whiteboard/screenshot/none
18
6. **Diagram analysis** -- High-confidence diagrams get full extraction (description, text, mermaid, chart data)
19
7. **Screengrab fallback** -- Medium-confidence frames are saved as captioned screenshots
20
8. **Knowledge graph** -- Extracts entities and relationships from transcript + diagrams, stored in both `knowledge_graph.db` (SQLite, primary) and `knowledge_graph.json` (export)
21
9. **Key points** -- LLM extracts main points and topics
22
10. **Action items** -- LLM finds tasks, commitments, and follow-ups
23
11. **Reports** -- Generates markdown, HTML, and PDF
24
12. **Export** -- Renders mermaid diagrams to SVG/PNG, reproduces charts
25
26
After analysis, you can optionally run planning taxonomy classification on the knowledge graph to categorize entities for use with the planning agent:
27
28
```bash
29
planopticon kg classify results/knowledge_graph.db
30
```
31
32
## Processing depth
33
34
### `basic`
35
- Transcription only
36
- Key points and action items
37
- No diagram extraction
38
39
### `standard` (default)
40
- Everything in basic
41
- Diagram extraction (up to 10 frames, evenly sampled)
42
- Knowledge graph
43
- Full report generation
44
45
### `comprehensive`
46
- Everything in standard
47
- More frames analyzed (up to 20)
48
- Deeper analysis
49
50
## Command-line options
51
52
### Provider and model selection
53
54
```bash
55
# Use a specific provider
56
planopticon analyze -i video.mp4 -o ./output --provider anthropic
57
58
# Override vision and chat models separately
59
planopticon analyze -i video.mp4 -o ./output --vision-model gpt-4o --chat-model claude-sonnet-4-20250514
60
```
61
62
### Speaker diarization hints
63
64
Use `--speakers` to provide speaker names as comma-separated hints. These are passed to the transcription provider to improve speaker identification in the transcript segments.
65
66
```bash
67
planopticon analyze -i video.mp4 -o ./output --speakers "Alice,Bob,Carol"
68
```
69
70
### Custom prompt templates
71
72
Use `--templates-dir` to point to a directory of custom `.txt` prompt template files. These override the built-in prompts used for diagram analysis, key point extraction, action item extraction, and other LLM-driven steps.
73
74
```bash
75
planopticon analyze -i video.mp4 -o ./output --templates-dir ./my-prompts
76
```
77
78
Template files should be named to match the built-in template names (e.g., `key_points.txt`, `action_items.txt`). See the `video_processor/utils/prompt_templates.py` module for the full list of template names.
79
80
### Output format
81
82
Use `--output-format json` to emit the complete `VideoManifest` as structured JSON to stdout, in addition to writing all output files to disk. This is useful for scripting, CI/CD integration, or piping results into other tools.
83
84
```bash
85
# Standard output (files + console summary)
86
planopticon analyze -i video.mp4 -o ./output
87
88
# JSON manifest to stdout
89
planopticon analyze -i video.mp4 -o ./output --output-format json
90
```
91
92
### Frame extraction tuning
93
94
```bash
95
# Adjust sampling rate (frames per second to consider)
96
planopticon analyze -i video.mp4 -o ./output --sampling-rate 1.0
97
98
# Adjust change detection threshold (lower = more sensitive)
99
planopticon analyze -i video.mp4 -o ./output --change-threshold 0.10
100
101
# Adjust periodic capture interval
102
planopticon analyze -i video.mp4 -o ./output --periodic-capture 60
103
104
# Enable GPU acceleration for frame extraction
105
planopticon analyze -i video.mp4 -o ./output --use-gpu
106
```
107
108
## Output structure
109
110
Every run produces a standardized directory structure:
111
112
```
113
output/
114
├── manifest.json # Run manifest (source of truth)
115
├── transcript/
116
│ ├── transcript.json # Full transcript with segments + speakers
117
│ ├── transcript.txt # Plain text
118
│ └── transcript.srt # Subtitles
119
├── frames/
120
│ ├── frame_0000.jpg
121
│ └── ...
122
├── diagrams/
123
│ ├── diagram_0.jpg # Original frame
124
│ ├── diagram_0.mermaid # Mermaid source
125
│ ├── diagram_0.svg # Vector rendering
126
│ ├── diagram_0.png # Raster rendering
127
│ ├── diagram_0.json # Analysis data
128
│ └── ...
129
├── captures/
130
│ ├── capture_0.jpg # Medium-confidence screenshots
131
│ ├── capture_0.json
132
│ └── ...
133
└── results/
134
├── analysis.md # Markdown report
135
├── analysis.html # HTML report
136
├── analysis.pdf # PDF (if planopticon[pdf] installed)
137
├── knowledge_graph.db # Knowledge graph (SQLite, primary)
138
├── knowledge_graph.json # Knowledge graph (JSON export)
139
├── key_points.json # Extracted key points
140
└── action_items.json # Action items
141
```
142
143
## Output manifest
144
145
Every run produces a `manifest.json` that is the single source of truth:
146
147
```json
148
{
149
"version": "1.0",
150
"video": {
151
"title": "Analysis of recording",
152
"source_path": "/path/to/recording.mp4",
153
"duration_seconds": 3600.0
154
},
155
"stats": {
156
"duration_seconds": 45.2,
157
"frames_extracted": 42,
158
"people_frames_filtered": 11,
159
"diagrams_detected": 3,
160
"screen_captures": 5,
161
"models_used": {
162
"vision": "gpt-4o",
163
"chat": "gpt-4o"
164
}
165
},
166
"transcript_json": "transcript/transcript.json",
167
"transcript_txt": "transcript/transcript.txt",
168
"transcript_srt": "transcript/transcript.srt",
169
"analysis_md": "results/analysis.md",
170
"knowledge_graph_json": "results/knowledge_graph.json",
171
"knowledge_graph_db": "results/knowledge_graph.db",
172
"key_points_json": "results/key_points.json",
173
"action_items_json": "results/action_items.json",
174
"key_points": [...],
175
"action_items": [...],
176
"diagrams": [...],
177
"screen_captures": [...]
178
}
179
```
180
181
## Checkpoint and resume
182
183
The pipeline supports checkpoint/resume. If a step's output files already exist on disk, that step is skipped on re-run. This means you can safely re-run an interrupted analysis and it will pick up where it left off:
184
185
```bash
186
# First run (interrupted at step 6)
187
planopticon analyze -i video.mp4 -o ./output
188
189
# Second run (resumes from step 6)
190
planopticon analyze -i video.mp4 -o ./output
191
```
192
193
## Using results after analysis
194
195
### Query the knowledge graph
196
197
After analysis completes, you can query the knowledge graph directly:
198
199
```bash
200
# Show graph stats
201
planopticon query --db results/knowledge_graph.db
202
203
# List entities by type
204
planopticon query --db results/knowledge_graph.db "entities --type technology"
205
206
# Find neighbors of an entity
207
planopticon query --db results/knowledge_graph.db "neighbors Kubernetes"
208
209
# Ask natural language questions (requires API key)
210
planopticon query --db results/knowledge_graph.db "What technologies were discussed?"
211
```
212
213
### Classify entities for planning
214
215
Run taxonomy classification to categorize entities into planning types (goal, milestone, risk, dependency, etc.):
216
217
```bash
218
planopticon kg classify results/knowledge_graph.db
219
planopticon kg classify results/knowledge_graph.db --format json
220
```
221
222
### Export to other formats
223
224
```bash
225
# Generate markdown documents
226
planopticon export markdown results/knowledge_graph.db -o ./docs
227
228
# Export as Obsidian vault
229
planopticon export obsidian results/knowledge_graph.db -o ./vault
230
231
# Export as PlanOpticonExchange
232
planopticon export exchange results/knowledge_graph.db -o exchange.json
233
234
# Generate GitHub wiki
235
planopticon wiki generate results/knowledge_graph.db -o ./wiki
236
```
237
238
### Use with the planning agent
239
240
The planning agent can consume the knowledge graph to generate project plans, PRDs, roadmaps, and other planning artifacts:
241
242
```bash
243
planopticon agent --db results/knowledge_graph.db
244
```
245

Keyboard Shortcuts

Open search /
Next entry (timeline) j
Previous entry (timeline) k
Open focused entry Enter
Show this help ?
Toggle theme Top nav button