Phase 1.1: 프로젝트 생성 — 폼 기반 도메인 선택 + 인라인 JSON 생성
백엔드 (crawler_platform/app/api/routes.py):
- POST /projects/inline 추가: 파일 시스템 의존 없이 JSON 본문으로 ProjectConfig 인라인 생성
- CreateProjectInlineRequest + InlineSourceConfig Pydantic 모델
- GET /domains 추가: 미리 정의된 도메인 목록 (perfume, tea, coffee, candle, supplement, gift)
- 각 도메인의 entity_types, predicates, attribute_count 반환
프론트엔드 API 레이어 (src/lib/api/):
- domains.ts: domainsApi.list + Zod 스키마
- ontology.ts: ontologyApi.get(domain) + Zod 스키마
- projects.ts: createInline() 메서드 + CreateProjectInlineRequest 타입
TanStack Query 훅 (src/hooks/):
- useDomains: 도메인 목록 (30분 staleTime)
- useOntology(domain): 도메인 상세
- useCreateProjectInline: 인라인 생성 mutation + projects 캐시 무효화
- queryKeys에 domains, ontology 키 팩토리 추가
UI 프리미티브 (src/components/ui/):
- input.tsx, label.tsx, textarea.tsx
OnboardingPage 완전 교체 (src/pages/OnboardingPage.tsx):
- react-hook-form + zod 검증 (project_name 정규식, 도메인 필수)
- 도메인 카드 그리드 선택 (entity/predicate 개수 미리보기)
- 백엔드 /projects/inline POST → 성공 시 sonner 토스트 + /sources/{name}으로 이동
- 로딩/에러/재시도 UI
기타:
- Vite proxy에 /domains prefix 추가
- i18n locale 키 onboarding.* 추가 (한/영)
다음 단계: Phase 1.2 — 온톨로지 엔티티/클레임 직접 입력 + URL 추출
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -7,7 +7,7 @@ from fastapi import BackgroundTasks, HTTPException
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy import select
|
||||
|
||||
from crawler_platform.app.config.loader import load_project_config
|
||||
from crawler_platform.app.config.loader import ProjectConfig, SourceConfig, load_project_config
|
||||
from crawler_platform.app.core.crawler.discovery import discover_links
|
||||
from crawler_platform.app.core.crawler.fetchers import RobotsPolicy, make_fetcher
|
||||
from crawler_platform.app.core.crawler.pipeline import CrawlPipeline
|
||||
@@ -17,7 +17,7 @@ from crawler_platform.app.core.database.repository import KnowledgeRepository
|
||||
from crawler_platform.app.core.database.session import session_scope
|
||||
from crawler_platform.app.core.extractor.ai_provider import list_openai_compatible_models
|
||||
from crawler_platform.app.core.extractor.factory import extractor_for_domain
|
||||
from crawler_platform.app.core.ontology.definitions import ontology_for_domain
|
||||
from crawler_platform.app.core.ontology.definitions import DOMAIN_ONTOLOGIES, ontology_for_domain
|
||||
from crawler_platform.app.core.ontology.gap_detector import KnowledgeGapDetector
|
||||
from crawler_platform.app.core.ontology.mapper import ontology_to_dict
|
||||
from crawler_platform.app.core.ontology.registry import OntologyRegistry
|
||||
@@ -65,6 +65,43 @@ class CreateProjectRequest(BaseModel):
|
||||
config_path: str
|
||||
|
||||
|
||||
class InlineSourceConfig(BaseModel):
|
||||
name: str
|
||||
type: str = "unknown"
|
||||
trust_level: float = 0.5
|
||||
base_url: str | None = None
|
||||
allowed_paths: list[str] = Field(default_factory=list)
|
||||
parser: str = "generic"
|
||||
fetcher: str = "requests"
|
||||
rate_limit_per_minute: int = 30
|
||||
respect_robots_txt: bool = False
|
||||
|
||||
|
||||
class CreateProjectInlineRequest(BaseModel):
|
||||
"""Create a project from inline JSON config (no filesystem dependency)."""
|
||||
|
||||
project_name: str
|
||||
domain: str
|
||||
target_entities: list[str] = Field(default_factory=list)
|
||||
fields: list[str] = Field(default_factory=list)
|
||||
sources: list[InlineSourceConfig] = Field(default_factory=list)
|
||||
ontology: dict[str, Any] = Field(default_factory=dict)
|
||||
recommendation: dict[str, Any] = Field(default_factory=dict)
|
||||
update_policy: dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
def to_project_config(self) -> ProjectConfig:
|
||||
return ProjectConfig(
|
||||
project_name=self.project_name,
|
||||
domain=self.domain,
|
||||
target_entities=list(self.target_entities),
|
||||
fields=list(self.fields),
|
||||
sources=[SourceConfig(**s.model_dump()) for s in self.sources],
|
||||
ontology=dict(self.ontology),
|
||||
recommendation=dict(self.recommendation),
|
||||
update_policy=dict(self.update_policy),
|
||||
)
|
||||
|
||||
|
||||
class ResetProjectRequest(BaseModel):
|
||||
config_path: str
|
||||
project_name: str | None = None
|
||||
@@ -298,6 +335,26 @@ def register_routes(app, database_url: str) -> None:
|
||||
project = KnowledgeRepository(session).upsert_project(config)
|
||||
return {"id": project.id, "name": project.name, "domain": project.domain}
|
||||
|
||||
@app.post("/projects/inline")
|
||||
def create_project_inline(request: CreateProjectInlineRequest):
|
||||
config = request.to_project_config()
|
||||
with session_scope(database_url) as session:
|
||||
project = KnowledgeRepository(session).upsert_project(config)
|
||||
return {"id": project.id, "name": project.name, "domain": project.domain}
|
||||
|
||||
@app.get("/domains")
|
||||
def list_domains():
|
||||
"""Available pre-defined ontology domains for project creation."""
|
||||
return [
|
||||
{
|
||||
"domain": ont.domain,
|
||||
"entity_types": list(ont.entity_types),
|
||||
"predicates": list(ont.predicates),
|
||||
"attribute_count": len(ont.attributes),
|
||||
}
|
||||
for ont in DOMAIN_ONTOLOGIES.values()
|
||||
]
|
||||
|
||||
@app.post("/projects/reset")
|
||||
def reset_project(request: ResetProjectRequest):
|
||||
config = load_project_config(request.config_path)
|
||||
|
||||
Reference in New Issue
Block a user