|
f0106a3…
|
leo
|
1 |
# Contributing |
|
f0106a3…
|
leo
|
2 |
|
|
f0106a3…
|
leo
|
3 |
## Development setup |
|
f0106a3…
|
leo
|
4 |
|
|
f0106a3…
|
leo
|
5 |
```bash |
|
c33fef2…
|
leo
|
6 |
git clone https://github.com/ConflictHQ/PlanOpticon.git |
|
f0106a3…
|
leo
|
7 |
cd PlanOpticon |
|
f0106a3…
|
leo
|
8 |
python -m venv .venv |
|
f0106a3…
|
leo
|
9 |
source .venv/bin/activate |
|
f0106a3…
|
leo
|
10 |
pip install -e ".[dev]" |
|
f0106a3…
|
leo
|
11 |
``` |
|
f0106a3…
|
leo
|
12 |
|
|
f0106a3…
|
leo
|
13 |
## Running tests |
|
c33fef2…
|
leo
|
14 |
|
|
3551b80…
|
noreply
|
15 |
PlanOpticon has 822+ tests covering providers, pipeline stages, document processors, knowledge graph operations, exporters, skills, and CLI commands. |
|
3551b80…
|
noreply
|
16 |
|
|
f0106a3…
|
leo
|
17 |
```bash |
|
f0106a3…
|
leo
|
18 |
# Run all tests |
|
f0106a3…
|
leo
|
19 |
pytest tests/ -v |
|
f0106a3…
|
leo
|
20 |
|
|
f0106a3…
|
leo
|
21 |
# Run with coverage |
|
f0106a3…
|
leo
|
22 |
pytest tests/ --cov=video_processor --cov-report=html |
|
f0106a3…
|
leo
|
23 |
|
|
f0106a3…
|
leo
|
24 |
# Run a specific test file |
|
f0106a3…
|
leo
|
25 |
pytest tests/test_models.py -v |
|
3551b80…
|
noreply
|
26 |
|
|
3551b80…
|
noreply
|
27 |
# Run tests matching a keyword |
|
3551b80…
|
noreply
|
28 |
pytest tests/ -k "test_knowledge_graph" -v |
|
3551b80…
|
noreply
|
29 |
|
|
3551b80…
|
noreply
|
30 |
# Run only fast tests (skip slow integration tests) |
|
3551b80…
|
noreply
|
31 |
pytest tests/ -m "not slow" -v |
|
3551b80…
|
noreply
|
32 |
``` |
|
3551b80…
|
noreply
|
33 |
|
|
3551b80…
|
noreply
|
34 |
### Test conventions |
|
3551b80…
|
noreply
|
35 |
|
|
3551b80…
|
noreply
|
36 |
- All tests live in the `tests/` directory, mirroring the `video_processor/` package structure |
|
3551b80…
|
noreply
|
37 |
- Test files are named `test_<module>.py` |
|
3551b80…
|
noreply
|
38 |
- Use `pytest` as the test runner -- do not use `unittest.TestCase` unless necessary for specific setup/teardown patterns |
|
3551b80…
|
noreply
|
39 |
- Mock external API calls. Never make real API calls in tests. Use `unittest.mock.patch` or `pytest-mock` fixtures to mock provider responses. |
|
3551b80…
|
noreply
|
40 |
- Use `tmp_path` (pytest fixture) for any tests that write files to disk |
|
3551b80…
|
noreply
|
41 |
- Fixtures shared across test files go in `conftest.py` |
|
3551b80…
|
noreply
|
42 |
- For testing CLI commands, use `click.testing.CliRunner` |
|
3551b80…
|
noreply
|
43 |
- For testing provider implementations, mock at the HTTP client level (e.g., patch `requests.post` or the provider's SDK client) |
|
3551b80…
|
noreply
|
44 |
|
|
3551b80…
|
noreply
|
45 |
### Mocking patterns |
|
3551b80…
|
noreply
|
46 |
|
|
3551b80…
|
noreply
|
47 |
```python |
|
3551b80…
|
noreply
|
48 |
# Mocking a provider's chat method |
|
3551b80…
|
noreply
|
49 |
from unittest.mock import MagicMock, patch |
|
3551b80…
|
noreply
|
50 |
|
|
3551b80…
|
noreply
|
51 |
def test_key_point_extraction(): |
|
3551b80…
|
noreply
|
52 |
pm = MagicMock() |
|
3551b80…
|
noreply
|
53 |
pm.chat.return_value = '["Point 1", "Point 2"]' |
|
3551b80…
|
noreply
|
54 |
result = extract_key_points(pm, "transcript text") |
|
3551b80…
|
noreply
|
55 |
assert len(result) == 2 |
|
3551b80…
|
noreply
|
56 |
|
|
3551b80…
|
noreply
|
57 |
# Mocking an external API at the HTTP level |
|
3551b80…
|
noreply
|
58 |
@patch("requests.post") |
|
3551b80…
|
noreply
|
59 |
def test_provider_chat(mock_post): |
|
3551b80…
|
noreply
|
60 |
mock_post.return_value.json.return_value = { |
|
3551b80…
|
noreply
|
61 |
"choices": [{"message": {"content": "response"}}] |
|
3551b80…
|
noreply
|
62 |
} |
|
3551b80…
|
noreply
|
63 |
provider = OpenAIProvider(api_key="test") |
|
3551b80…
|
noreply
|
64 |
result = provider.chat([{"role": "user", "content": "hello"}]) |
|
3551b80…
|
noreply
|
65 |
assert result == "response" |
|
f0106a3…
|
leo
|
66 |
``` |
|
f0106a3…
|
leo
|
67 |
|
|
f0106a3…
|
leo
|
68 |
## Code style |
|
f0106a3…
|
leo
|
69 |
|
|
f0106a3…
|
leo
|
70 |
We use: |
|
f0106a3…
|
leo
|
71 |
|
|
3551b80…
|
noreply
|
72 |
- **Ruff** for both linting and formatting (100 char line length) |
|
f0106a3…
|
leo
|
73 |
- **mypy** for type checking |
|
3551b80…
|
noreply
|
74 |
|
|
3551b80…
|
noreply
|
75 |
Ruff handles all linting (error, warning, pyflakes, and import sorting rules) and formatting in a single tool. There is no need to run Black or isort separately. |
|
f0106a3…
|
leo
|
76 |
|
|
f0106a3…
|
leo
|
77 |
```bash |
|
3551b80…
|
noreply
|
78 |
# Lint |
|
f0106a3…
|
leo
|
79 |
ruff check video_processor/ |
|
3551b80…
|
noreply
|
80 |
|
|
3551b80…
|
noreply
|
81 |
# Format |
|
3551b80…
|
noreply
|
82 |
ruff format video_processor/ |
|
3551b80…
|
noreply
|
83 |
|
|
3551b80…
|
noreply
|
84 |
# Auto-fix lint issues |
|
3551b80…
|
noreply
|
85 |
ruff check video_processor/ --fix |
|
3551b80…
|
noreply
|
86 |
|
|
3551b80…
|
noreply
|
87 |
# Type check |
|
f0106a3…
|
leo
|
88 |
mypy video_processor/ --ignore-missing-imports |
|
f0106a3…
|
leo
|
89 |
``` |
|
f0106a3…
|
leo
|
90 |
|
|
3551b80…
|
noreply
|
91 |
### Ruff configuration |
|
3551b80…
|
noreply
|
92 |
|
|
3551b80…
|
noreply
|
93 |
The project's `pyproject.toml` configures ruff as follows: |
|
3551b80…
|
noreply
|
94 |
|
|
3551b80…
|
noreply
|
95 |
```toml |
|
3551b80…
|
noreply
|
96 |
[tool.ruff] |
|
3551b80…
|
noreply
|
97 |
line-length = 100 |
|
3551b80…
|
noreply
|
98 |
target-version = "py310" |
|
3551b80…
|
noreply
|
99 |
|
|
3551b80…
|
noreply
|
100 |
[tool.ruff.lint] |
|
3551b80…
|
noreply
|
101 |
select = ["E", "F", "W", "I"] |
|
3551b80…
|
noreply
|
102 |
``` |
|
3551b80…
|
noreply
|
103 |
|
|
3551b80…
|
noreply
|
104 |
The `I` rule set covers import sorting (equivalent to isort), so imports are automatically organized by ruff. |
|
3551b80…
|
noreply
|
105 |
|
|
f0106a3…
|
leo
|
106 |
## Project structure |
|
f0106a3…
|
leo
|
107 |
|
|
3551b80…
|
noreply
|
108 |
``` |
|
3551b80…
|
noreply
|
109 |
PlanOpticon/ |
|
3551b80…
|
noreply
|
110 |
├── video_processor/ |
|
3551b80…
|
noreply
|
111 |
│ ├── cli/ # Click CLI commands |
|
3551b80…
|
noreply
|
112 |
│ │ └── commands.py |
|
3551b80…
|
noreply
|
113 |
│ ├── providers/ # LLM/API provider implementations |
|
3551b80…
|
noreply
|
114 |
│ │ ├── base.py # BaseProvider, ProviderRegistry |
|
3551b80…
|
noreply
|
115 |
│ │ ├── manager.py # ProviderManager |
|
3551b80…
|
noreply
|
116 |
│ │ ├── discovery.py # Auto-discovery of available providers |
|
3551b80…
|
noreply
|
117 |
│ │ ├── openai_provider.py |
|
3551b80…
|
noreply
|
118 |
│ │ ├── anthropic_provider.py |
|
3551b80…
|
noreply
|
119 |
│ │ ├── gemini_provider.py |
|
3551b80…
|
noreply
|
120 |
│ │ └── ... # 15+ provider implementations |
|
3551b80…
|
noreply
|
121 |
│ ├── sources/ # Cloud and web source connectors |
|
3551b80…
|
noreply
|
122 |
│ │ ├── base.py # BaseSource, SourceFile |
|
3551b80…
|
noreply
|
123 |
│ │ ├── google_drive.py |
|
3551b80…
|
noreply
|
124 |
│ │ ├── zoom_source.py |
|
3551b80…
|
noreply
|
125 |
│ │ └── ... # 20+ source implementations |
|
3551b80…
|
noreply
|
126 |
│ ├── processors/ # Document processors |
|
3551b80…
|
noreply
|
127 |
│ │ ├── base.py # DocumentProcessor, registry |
|
3551b80…
|
noreply
|
128 |
│ │ ├── ingest.py # File/directory ingestion |
|
3551b80…
|
noreply
|
129 |
│ │ ├── markdown_processor.py |
|
3551b80…
|
noreply
|
130 |
│ │ ├── pdf_processor.py |
|
3551b80…
|
noreply
|
131 |
│ │ └── __init__.py # Auto-registration of built-in processors |
|
3551b80…
|
noreply
|
132 |
│ ├── integrators/ # Knowledge graph and analysis |
|
3551b80…
|
noreply
|
133 |
│ │ ├── knowledge_graph.py # KnowledgeGraph class |
|
3551b80…
|
noreply
|
134 |
│ │ ├── graph_store.py # SQLite graph storage |
|
3551b80…
|
noreply
|
135 |
│ │ ├── graph_query.py # GraphQueryEngine |
|
3551b80…
|
noreply
|
136 |
│ │ ├── graph_discovery.py # Auto-find knowledge_graph.db |
|
3551b80…
|
noreply
|
137 |
│ │ └── taxonomy.py # Planning taxonomy classifier |
|
3551b80…
|
noreply
|
138 |
│ ├── agent/ # Planning agent |
|
3551b80…
|
noreply
|
139 |
│ │ ├── orchestrator.py # Agent orchestration |
|
3551b80…
|
noreply
|
140 |
│ │ └── skills/ # Skill implementations |
|
3551b80…
|
noreply
|
141 |
│ │ ├── base.py # Skill ABC, registry, Artifact |
|
3551b80…
|
noreply
|
142 |
│ │ ├── project_plan.py |
|
3551b80…
|
noreply
|
143 |
│ │ ├── prd.py |
|
3551b80…
|
noreply
|
144 |
│ │ ├── roadmap.py |
|
3551b80…
|
noreply
|
145 |
│ │ ├── task_breakdown.py |
|
3551b80…
|
noreply
|
146 |
│ │ ├── doc_generator.py |
|
3551b80…
|
noreply
|
147 |
│ │ ├── wiki_generator.py |
|
3551b80…
|
noreply
|
148 |
│ │ ├── notes_export.py |
|
3551b80…
|
noreply
|
149 |
│ │ ├── artifact_export.py |
|
3551b80…
|
noreply
|
150 |
│ │ ├── github_integration.py |
|
3551b80…
|
noreply
|
151 |
│ │ ├── requirements_chat.py |
|
3551b80…
|
noreply
|
152 |
│ │ ├── cli_adapter.py |
|
3551b80…
|
noreply
|
153 |
│ │ └── __init__.py # Auto-registration of skills |
|
3551b80…
|
noreply
|
154 |
│ ├── exporters/ # Output format exporters |
|
3551b80…
|
noreply
|
155 |
│ │ ├── __init__.py |
|
3551b80…
|
noreply
|
156 |
│ │ └── markdown.py # Template-based markdown generation |
|
3551b80…
|
noreply
|
157 |
│ ├── utils/ # Shared utilities |
|
3551b80…
|
noreply
|
158 |
│ │ ├── export.py # Multi-format export orchestration |
|
3551b80…
|
noreply
|
159 |
│ │ ├── rendering.py # Mermaid/chart rendering |
|
3551b80…
|
noreply
|
160 |
│ │ ├── prompt_templates.py |
|
3551b80…
|
noreply
|
161 |
│ │ ├── callbacks.py # Progress callback helpers |
|
3551b80…
|
noreply
|
162 |
│ │ └── ... |
|
3551b80…
|
noreply
|
163 |
│ ├── exchange.py # PlanOpticonExchange format |
|
3551b80…
|
noreply
|
164 |
│ ├── pipeline.py # Main video processing pipeline |
|
3551b80…
|
noreply
|
165 |
│ ├── models.py # Pydantic data models |
|
3551b80…
|
noreply
|
166 |
│ └── output_structure.py # Output directory helpers |
|
3551b80…
|
noreply
|
167 |
├── tests/ # 822+ tests |
|
3551b80…
|
noreply
|
168 |
├── knowledge-base/ # Local-first graph tools |
|
3551b80…
|
noreply
|
169 |
│ ├── viewer.html # Self-contained D3.js graph viewer |
|
3551b80…
|
noreply
|
170 |
│ └── query.py # Python query script (NetworkX) |
|
3551b80…
|
noreply
|
171 |
├── docs/ # MkDocs documentation |
|
3551b80…
|
noreply
|
172 |
└── pyproject.toml # Project configuration |
|
3551b80…
|
noreply
|
173 |
``` |
|
3551b80…
|
noreply
|
174 |
|
|
3551b80…
|
noreply
|
175 |
See [Architecture Overview](architecture/overview.md) for a more detailed breakdown of module responsibilities. |
|
f0106a3…
|
leo
|
176 |
|
|
f0106a3…
|
leo
|
177 |
## Adding a new provider |
|
f0106a3…
|
leo
|
178 |
|
|
3551b80…
|
noreply
|
179 |
Providers self-register via `ProviderRegistry.register()` at module level. When the provider module is imported, it registers itself automatically. |
|
3551b80…
|
noreply
|
180 |
|
|
f0106a3…
|
leo
|
181 |
1. Create `video_processor/providers/your_provider.py` |
|
f0106a3…
|
leo
|
182 |
2. Extend `BaseProvider` from `video_processor/providers/base.py` |
|
3551b80…
|
noreply
|
183 |
3. Implement the four required methods: `chat()`, `analyze_image()`, `transcribe_audio()`, `list_models()` |
|
3551b80…
|
noreply
|
184 |
4. Call `ProviderRegistry.register()` at module level |
|
3551b80…
|
noreply
|
185 |
5. Add the import to `video_processor/providers/manager.py` in the lazy-import block |
|
3551b80…
|
noreply
|
186 |
6. Add tests in `tests/test_providers.py` |
|
3551b80…
|
noreply
|
187 |
|
|
3551b80…
|
noreply
|
188 |
### Example provider skeleton |
|
3551b80…
|
noreply
|
189 |
|
|
3551b80…
|
noreply
|
190 |
```python |
|
3551b80…
|
noreply
|
191 |
"""Your provider implementation.""" |
|
3551b80…
|
noreply
|
192 |
|
|
3551b80…
|
noreply
|
193 |
from video_processor.providers.base import BaseProvider, ModelInfo, ProviderRegistry |
|
3551b80…
|
noreply
|
194 |
|
|
3551b80…
|
noreply
|
195 |
|
|
3551b80…
|
noreply
|
196 |
class YourProvider(BaseProvider): |
|
3551b80…
|
noreply
|
197 |
provider_name = "yourprovider" |
|
3551b80…
|
noreply
|
198 |
|
|
3551b80…
|
noreply
|
199 |
def __init__(self, api_key: str | None = None): |
|
3551b80…
|
noreply
|
200 |
import os |
|
3551b80…
|
noreply
|
201 |
self.api_key = api_key or os.environ.get("YOUR_API_KEY", "") |
|
3551b80…
|
noreply
|
202 |
|
|
3551b80…
|
noreply
|
203 |
def chat(self, messages, max_tokens=4096, temperature=0.7, model=None): |
|
3551b80…
|
noreply
|
204 |
# Implement chat completion |
|
3551b80…
|
noreply
|
205 |
... |
|
3551b80…
|
noreply
|
206 |
|
|
3551b80…
|
noreply
|
207 |
def analyze_image(self, image_bytes, prompt, max_tokens=4096, model=None): |
|
3551b80…
|
noreply
|
208 |
# Implement image analysis |
|
3551b80…
|
noreply
|
209 |
... |
|
3551b80…
|
noreply
|
210 |
|
|
3551b80…
|
noreply
|
211 |
def transcribe_audio(self, audio_path, language=None, model=None): |
|
3551b80…
|
noreply
|
212 |
# Implement audio transcription (or raise NotImplementedError) |
|
3551b80…
|
noreply
|
213 |
... |
|
3551b80…
|
noreply
|
214 |
|
|
3551b80…
|
noreply
|
215 |
def list_models(self): |
|
3551b80…
|
noreply
|
216 |
return [ModelInfo(id="your-model", provider="yourprovider", capabilities=["chat"])] |
|
3551b80…
|
noreply
|
217 |
|
|
3551b80…
|
noreply
|
218 |
|
|
3551b80…
|
noreply
|
219 |
# Self-registration at import time |
|
3551b80…
|
noreply
|
220 |
ProviderRegistry.register( |
|
3551b80…
|
noreply
|
221 |
"yourprovider", |
|
3551b80…
|
noreply
|
222 |
YourProvider, |
|
3551b80…
|
noreply
|
223 |
env_var="YOUR_API_KEY", |
|
3551b80…
|
noreply
|
224 |
model_prefixes=["your-"], |
|
3551b80…
|
noreply
|
225 |
default_models={"chat": "your-model"}, |
|
3551b80…
|
noreply
|
226 |
) |
|
3551b80…
|
noreply
|
227 |
``` |
|
3551b80…
|
noreply
|
228 |
|
|
3551b80…
|
noreply
|
229 |
### OpenAI-compatible providers |
|
3551b80…
|
noreply
|
230 |
|
|
3551b80…
|
noreply
|
231 |
For providers that use the OpenAI API format, extend `OpenAICompatibleProvider` instead of `BaseProvider`. This provides default implementations of `chat()`, `analyze_image()`, and `list_models()` -- you only need to configure the base URL and model mappings. |
|
3551b80…
|
noreply
|
232 |
|
|
3551b80…
|
noreply
|
233 |
```python |
|
3551b80…
|
noreply
|
234 |
from video_processor.providers.base import OpenAICompatibleProvider, ProviderRegistry |
|
3551b80…
|
noreply
|
235 |
|
|
3551b80…
|
noreply
|
236 |
class YourProvider(OpenAICompatibleProvider): |
|
3551b80…
|
noreply
|
237 |
provider_name = "yourprovider" |
|
3551b80…
|
noreply
|
238 |
base_url = "https://api.yourprovider.com/v1" |
|
3551b80…
|
noreply
|
239 |
env_var = "YOUR_API_KEY" |
|
3551b80…
|
noreply
|
240 |
|
|
3551b80…
|
noreply
|
241 |
ProviderRegistry.register("yourprovider", YourProvider, env_var="YOUR_API_KEY") |
|
3551b80…
|
noreply
|
242 |
``` |
|
f0106a3…
|
leo
|
243 |
|
|
f0106a3…
|
leo
|
244 |
## Adding a new cloud source |
|
f0106a3…
|
leo
|
245 |
|
|
3551b80…
|
noreply
|
246 |
Source connectors implement the `BaseSource` ABC from `video_processor/sources/base.py`. Authentication is handled per-source, typically via environment variables. |
|
3551b80…
|
noreply
|
247 |
|
|
f0106a3…
|
leo
|
248 |
1. Create `video_processor/sources/your_source.py` |
|
3551b80…
|
noreply
|
249 |
2. Extend `BaseSource` |
|
3551b80…
|
noreply
|
250 |
3. Implement `authenticate()`, `list_videos()`, and `download()` |
|
3551b80…
|
noreply
|
251 |
4. Add the class to the lazy-import map in `video_processor/sources/__init__.py` |
|
3551b80…
|
noreply
|
252 |
5. Add CLI commands in `video_processor/cli/commands.py` if needed |
|
3551b80…
|
noreply
|
253 |
6. Add tests and documentation |
|
3551b80…
|
noreply
|
254 |
|
|
3551b80…
|
noreply
|
255 |
### Example source skeleton |
|
3551b80…
|
noreply
|
256 |
|
|
3551b80…
|
noreply
|
257 |
```python |
|
3551b80…
|
noreply
|
258 |
"""Your source integration.""" |
|
3551b80…
|
noreply
|
259 |
|
|
3551b80…
|
noreply
|
260 |
import os |
|
3551b80…
|
noreply
|
261 |
import logging |
|
3551b80…
|
noreply
|
262 |
from pathlib import Path |
|
3551b80…
|
noreply
|
263 |
from typing import List, Optional |
|
3551b80…
|
noreply
|
264 |
|
|
3551b80…
|
noreply
|
265 |
from video_processor.sources.base import BaseSource, SourceFile |
|
3551b80…
|
noreply
|
266 |
|
|
3551b80…
|
noreply
|
267 |
logger = logging.getLogger(__name__) |
|
3551b80…
|
noreply
|
268 |
|
|
3551b80…
|
noreply
|
269 |
|
|
3551b80…
|
noreply
|
270 |
class YourSource(BaseSource): |
|
3551b80…
|
noreply
|
271 |
def __init__(self, api_key: Optional[str] = None): |
|
3551b80…
|
noreply
|
272 |
self.api_key = api_key or os.environ.get("YOUR_SOURCE_KEY", "") |
|
3551b80…
|
noreply
|
273 |
|
|
3551b80…
|
noreply
|
274 |
def authenticate(self) -> bool: |
|
3551b80…
|
noreply
|
275 |
"""Validate credentials. Return True on success.""" |
|
3551b80…
|
noreply
|
276 |
if not self.api_key: |
|
3551b80…
|
noreply
|
277 |
logger.error("API key not set. Set YOUR_SOURCE_KEY env var.") |
|
3551b80…
|
noreply
|
278 |
return False |
|
3551b80…
|
noreply
|
279 |
# Make a test API call to verify credentials |
|
3551b80…
|
noreply
|
280 |
... |
|
3551b80…
|
noreply
|
281 |
return True |
|
3551b80…
|
noreply
|
282 |
|
|
3551b80…
|
noreply
|
283 |
def list_videos( |
|
3551b80…
|
noreply
|
284 |
self, |
|
3551b80…
|
noreply
|
285 |
folder_id: Optional[str] = None, |
|
3551b80…
|
noreply
|
286 |
folder_path: Optional[str] = None, |
|
3551b80…
|
noreply
|
287 |
patterns: Optional[List[str]] = None, |
|
3551b80…
|
noreply
|
288 |
) -> List[SourceFile]: |
|
3551b80…
|
noreply
|
289 |
"""List available video files.""" |
|
3551b80…
|
noreply
|
290 |
... |
|
3551b80…
|
noreply
|
291 |
|
|
3551b80…
|
noreply
|
292 |
def download(self, file: SourceFile, destination: Path) -> Path: |
|
3551b80…
|
noreply
|
293 |
"""Download a single file. Return the local path.""" |
|
3551b80…
|
noreply
|
294 |
destination.parent.mkdir(parents=True, exist_ok=True) |
|
3551b80…
|
noreply
|
295 |
# Download file content to destination |
|
3551b80…
|
noreply
|
296 |
... |
|
3551b80…
|
noreply
|
297 |
return destination |
|
3551b80…
|
noreply
|
298 |
``` |
|
3551b80…
|
noreply
|
299 |
|
|
3551b80…
|
noreply
|
300 |
### Registering in `__init__.py` |
|
3551b80…
|
noreply
|
301 |
|
|
3551b80…
|
noreply
|
302 |
Add your source to the `__all__` list and the `_lazy_map` dictionary in `video_processor/sources/__init__.py`: |
|
3551b80…
|
noreply
|
303 |
|
|
3551b80…
|
noreply
|
304 |
```python |
|
3551b80…
|
noreply
|
305 |
__all__ = [ |
|
3551b80…
|
noreply
|
306 |
... |
|
3551b80…
|
noreply
|
307 |
"YourSource", |
|
3551b80…
|
noreply
|
308 |
] |
|
3551b80…
|
noreply
|
309 |
|
|
3551b80…
|
noreply
|
310 |
_lazy_map = { |
|
3551b80…
|
noreply
|
311 |
... |
|
3551b80…
|
noreply
|
312 |
"YourSource": "video_processor.sources.your_source", |
|
3551b80…
|
noreply
|
313 |
} |
|
3551b80…
|
noreply
|
314 |
``` |
|
3551b80…
|
noreply
|
315 |
|
|
3551b80…
|
noreply
|
316 |
## Adding a new skill |
|
3551b80…
|
noreply
|
317 |
|
|
3551b80…
|
noreply
|
318 |
Agent skills extend the `Skill` ABC from `video_processor/agent/skills/base.py` and self-register via `register_skill()`. |
|
3551b80…
|
noreply
|
319 |
|
|
3551b80…
|
noreply
|
320 |
1. Create `video_processor/agent/skills/your_skill.py` |
|
3551b80…
|
noreply
|
321 |
2. Extend `Skill` and set `name` and `description` class attributes |
|
3551b80…
|
noreply
|
322 |
3. Implement `execute()` to return an `Artifact` |
|
3551b80…
|
noreply
|
323 |
4. Optionally override `can_execute()` for custom precondition checks |
|
3551b80…
|
noreply
|
324 |
5. Call `register_skill()` at module level |
|
3551b80…
|
noreply
|
325 |
6. Add the import to `video_processor/agent/skills/__init__.py` |
|
3551b80…
|
noreply
|
326 |
7. Add tests |
|
3551b80…
|
noreply
|
327 |
|
|
3551b80…
|
noreply
|
328 |
### Example skill skeleton |
|
3551b80…
|
noreply
|
329 |
|
|
3551b80…
|
noreply
|
330 |
```python |
|
3551b80…
|
noreply
|
331 |
"""Your custom skill.""" |
|
3551b80…
|
noreply
|
332 |
|
|
3551b80…
|
noreply
|
333 |
from video_processor.agent.skills.base import AgentContext, Artifact, Skill, register_skill |
|
3551b80…
|
noreply
|
334 |
|
|
3551b80…
|
noreply
|
335 |
|
|
3551b80…
|
noreply
|
336 |
class YourSkill(Skill): |
|
3551b80…
|
noreply
|
337 |
name = "your_skill" |
|
3551b80…
|
noreply
|
338 |
description = "Generates a custom artifact from the knowledge graph." |
|
3551b80…
|
noreply
|
339 |
|
|
3551b80…
|
noreply
|
340 |
def execute(self, context: AgentContext, **kwargs) -> Artifact: |
|
3551b80…
|
noreply
|
341 |
"""Generate the artifact.""" |
|
3551b80…
|
noreply
|
342 |
kg_data = context.knowledge_graph.to_dict() |
|
3551b80…
|
noreply
|
343 |
# Build content from knowledge graph data |
|
3551b80…
|
noreply
|
344 |
content = f"# Your Artifact\n\n{len(kg_data.get('entities', []))} entities found." |
|
3551b80…
|
noreply
|
345 |
return Artifact( |
|
3551b80…
|
noreply
|
346 |
name="your_artifact", |
|
3551b80…
|
noreply
|
347 |
content=content, |
|
3551b80…
|
noreply
|
348 |
artifact_type="document", |
|
3551b80…
|
noreply
|
349 |
format="markdown", |
|
3551b80…
|
noreply
|
350 |
) |
|
3551b80…
|
noreply
|
351 |
|
|
3551b80…
|
noreply
|
352 |
def can_execute(self, context: AgentContext) -> bool: |
|
3551b80…
|
noreply
|
353 |
"""Check prerequisites (default requires KG + provider).""" |
|
3551b80…
|
noreply
|
354 |
return context.knowledge_graph is not None |
|
3551b80…
|
noreply
|
355 |
|
|
3551b80…
|
noreply
|
356 |
|
|
3551b80…
|
noreply
|
357 |
# Self-registration at import time |
|
3551b80…
|
noreply
|
358 |
register_skill(YourSkill()) |
|
3551b80…
|
noreply
|
359 |
``` |
|
3551b80…
|
noreply
|
360 |
|
|
3551b80…
|
noreply
|
361 |
### Registering in `__init__.py` |
|
3551b80…
|
noreply
|
362 |
|
|
3551b80…
|
noreply
|
363 |
Add the import to `video_processor/agent/skills/__init__.py` so the skill is loaded (and self-registered) when the skills package is imported: |
|
3551b80…
|
noreply
|
364 |
|
|
3551b80…
|
noreply
|
365 |
```python |
|
3551b80…
|
noreply
|
366 |
from video_processor.agent.skills import ( |
|
3551b80…
|
noreply
|
367 |
... |
|
3551b80…
|
noreply
|
368 |
your_skill, # noqa: F401 |
|
3551b80…
|
noreply
|
369 |
) |
|
3551b80…
|
noreply
|
370 |
``` |
|
3551b80…
|
noreply
|
371 |
|
|
3551b80…
|
noreply
|
372 |
## Adding a new document processor |
|
3551b80…
|
noreply
|
373 |
|
|
3551b80…
|
noreply
|
374 |
Document processors extend the `DocumentProcessor` ABC from `video_processor/processors/base.py` and are registered via `register_processor()`. |
|
3551b80…
|
noreply
|
375 |
|
|
3551b80…
|
noreply
|
376 |
1. Create `video_processor/processors/your_processor.py` |
|
3551b80…
|
noreply
|
377 |
2. Extend `DocumentProcessor` |
|
3551b80…
|
noreply
|
378 |
3. Set `supported_extensions` class attribute |
|
3551b80…
|
noreply
|
379 |
4. Implement `process()` (returns `List[DocumentChunk]`) and `can_process()` |
|
3551b80…
|
noreply
|
380 |
5. Call `register_processor()` at module level |
|
3551b80…
|
noreply
|
381 |
6. Add the import to `video_processor/processors/__init__.py` |
|
3551b80…
|
noreply
|
382 |
7. Add tests |
|
3551b80…
|
noreply
|
383 |
|
|
3551b80…
|
noreply
|
384 |
### Example processor skeleton |
|
3551b80…
|
noreply
|
385 |
|
|
3551b80…
|
noreply
|
386 |
```python |
|
3551b80…
|
noreply
|
387 |
"""Your document processor.""" |
|
3551b80…
|
noreply
|
388 |
|
|
3551b80…
|
noreply
|
389 |
from pathlib import Path |
|
3551b80…
|
noreply
|
390 |
from typing import List |
|
3551b80…
|
noreply
|
391 |
|
|
3551b80…
|
noreply
|
392 |
from video_processor.processors.base import ( |
|
3551b80…
|
noreply
|
393 |
DocumentChunk, |
|
3551b80…
|
noreply
|
394 |
DocumentProcessor, |
|
3551b80…
|
noreply
|
395 |
register_processor, |
|
3551b80…
|
noreply
|
396 |
) |
|
3551b80…
|
noreply
|
397 |
|
|
3551b80…
|
noreply
|
398 |
|
|
3551b80…
|
noreply
|
399 |
class YourProcessor(DocumentProcessor): |
|
3551b80…
|
noreply
|
400 |
supported_extensions = [".xyz", ".abc"] |
|
3551b80…
|
noreply
|
401 |
|
|
3551b80…
|
noreply
|
402 |
def can_process(self, path: Path) -> bool: |
|
3551b80…
|
noreply
|
403 |
return path.suffix.lower() in self.supported_extensions |
|
3551b80…
|
noreply
|
404 |
|
|
3551b80…
|
noreply
|
405 |
def process(self, path: Path) -> List[DocumentChunk]: |
|
3551b80…
|
noreply
|
406 |
text = path.read_text() |
|
3551b80…
|
noreply
|
407 |
# Split into chunks as appropriate for your format |
|
3551b80…
|
noreply
|
408 |
return [ |
|
3551b80…
|
noreply
|
409 |
DocumentChunk( |
|
3551b80…
|
noreply
|
410 |
text=text, |
|
3551b80…
|
noreply
|
411 |
source_file=str(path), |
|
3551b80…
|
noreply
|
412 |
chunk_index=0, |
|
3551b80…
|
noreply
|
413 |
metadata={"format": "xyz"}, |
|
3551b80…
|
noreply
|
414 |
) |
|
3551b80…
|
noreply
|
415 |
] |
|
3551b80…
|
noreply
|
416 |
|
|
3551b80…
|
noreply
|
417 |
|
|
3551b80…
|
noreply
|
418 |
# Self-registration at import time |
|
3551b80…
|
noreply
|
419 |
register_processor([".xyz", ".abc"], YourProcessor) |
|
3551b80…
|
noreply
|
420 |
``` |
|
3551b80…
|
noreply
|
421 |
|
|
3551b80…
|
noreply
|
422 |
### Registering in `__init__.py` |
|
3551b80…
|
noreply
|
423 |
|
|
3551b80…
|
noreply
|
424 |
Add the import to `video_processor/processors/__init__.py`: |
|
3551b80…
|
noreply
|
425 |
|
|
3551b80…
|
noreply
|
426 |
```python |
|
3551b80…
|
noreply
|
427 |
from video_processor.processors import ( |
|
3551b80…
|
noreply
|
428 |
markdown_processor, # noqa: F401, E402 |
|
3551b80…
|
noreply
|
429 |
pdf_processor, # noqa: F401, E402 |
|
3551b80…
|
noreply
|
430 |
your_processor, # noqa: F401, E402 |
|
3551b80…
|
noreply
|
431 |
) |
|
3551b80…
|
noreply
|
432 |
``` |
|
3551b80…
|
noreply
|
433 |
|
|
3551b80…
|
noreply
|
434 |
## Adding a new exporter |
|
3551b80…
|
noreply
|
435 |
|
|
3551b80…
|
noreply
|
436 |
Exporters live in `video_processor/exporters/` and are typically called from CLI commands. There is no strict ABC for exporters -- they are plain functions that accept knowledge graph data and an output directory. |
|
3551b80…
|
noreply
|
437 |
|
|
3551b80…
|
noreply
|
438 |
1. Create `video_processor/exporters/your_exporter.py` |
|
3551b80…
|
noreply
|
439 |
2. Implement one or more export functions that accept KG data (as a dict) and an output path |
|
3551b80…
|
noreply
|
440 |
3. Add CLI integration in `video_processor/cli/commands.py` under the `export` group |
|
3551b80…
|
noreply
|
441 |
4. Add tests |
|
3551b80…
|
noreply
|
442 |
|
|
3551b80…
|
noreply
|
443 |
### Example exporter skeleton |
|
3551b80…
|
noreply
|
444 |
|
|
3551b80…
|
noreply
|
445 |
```python |
|
3551b80…
|
noreply
|
446 |
"""Your exporter.""" |
|
3551b80…
|
noreply
|
447 |
|
|
3551b80…
|
noreply
|
448 |
import json |
|
3551b80…
|
noreply
|
449 |
from pathlib import Path |
|
3551b80…
|
noreply
|
450 |
from typing import List |
|
3551b80…
|
noreply
|
451 |
|
|
3551b80…
|
noreply
|
452 |
|
|
3551b80…
|
noreply
|
453 |
def export_your_format(kg_data: dict, output_dir: Path) -> List[Path]: |
|
3551b80…
|
noreply
|
454 |
"""Export knowledge graph data in your format. |
|
3551b80…
|
noreply
|
455 |
|
|
3551b80…
|
noreply
|
456 |
Args: |
|
3551b80…
|
noreply
|
457 |
kg_data: Knowledge graph as a dict (from KnowledgeGraph.to_dict()). |
|
3551b80…
|
noreply
|
458 |
output_dir: Directory to write output files. |
|
3551b80…
|
noreply
|
459 |
|
|
3551b80…
|
noreply
|
460 |
Returns: |
|
3551b80…
|
noreply
|
461 |
List of created file paths. |
|
3551b80…
|
noreply
|
462 |
""" |
|
3551b80…
|
noreply
|
463 |
output_dir.mkdir(parents=True, exist_ok=True) |
|
3551b80…
|
noreply
|
464 |
created = [] |
|
3551b80…
|
noreply
|
465 |
|
|
3551b80…
|
noreply
|
466 |
output_file = output_dir / "export.xyz" |
|
3551b80…
|
noreply
|
467 |
output_file.write_text(json.dumps(kg_data, indent=2)) |
|
3551b80…
|
noreply
|
468 |
created.append(output_file) |
|
3551b80…
|
noreply
|
469 |
|
|
3551b80…
|
noreply
|
470 |
return created |
|
3551b80…
|
noreply
|
471 |
``` |
|
3551b80…
|
noreply
|
472 |
|
|
3551b80…
|
noreply
|
473 |
### Adding the CLI command |
|
3551b80…
|
noreply
|
474 |
|
|
3551b80…
|
noreply
|
475 |
Add a subcommand under the `export` group in `video_processor/cli/commands.py`: |
|
3551b80…
|
noreply
|
476 |
|
|
3551b80…
|
noreply
|
477 |
```python |
|
3551b80…
|
noreply
|
478 |
@export.command("your-format") |
|
3551b80…
|
noreply
|
479 |
@click.argument("db_path", type=click.Path(exists=True)) |
|
3551b80…
|
noreply
|
480 |
@click.option("-o", "--output", type=click.Path(), default=None) |
|
3551b80…
|
noreply
|
481 |
def export_your_format_cmd(db_path, output): |
|
3551b80…
|
noreply
|
482 |
"""Export knowledge graph in your format.""" |
|
3551b80…
|
noreply
|
483 |
from video_processor.exporters.your_exporter import export_your_format |
|
3551b80…
|
noreply
|
484 |
from video_processor.integrators.knowledge_graph import KnowledgeGraph |
|
3551b80…
|
noreply
|
485 |
|
|
3551b80…
|
noreply
|
486 |
kg = KnowledgeGraph(db_path=Path(db_path)) |
|
3551b80…
|
noreply
|
487 |
out_dir = Path(output) if output else Path.cwd() / "your-export" |
|
3551b80…
|
noreply
|
488 |
created = export_your_format(kg.to_dict(), out_dir) |
|
3551b80…
|
noreply
|
489 |
click.echo(f"Exported {len(created)} files to {out_dir}/") |
|
3551b80…
|
noreply
|
490 |
``` |
|
f0106a3…
|
leo
|
491 |
|
|
f0106a3…
|
leo
|
492 |
## License |
|
f0106a3…
|
leo
|
493 |
|
|
3551b80…
|
noreply
|
494 |
MIT License -- Copyright (c) 2026 CONFLICT LLC. All rights reserved. |