mirror of
https://github.com/sveltejs/ai-tools.git
synced 2026-07-04 03:19:38 +08:00
Compare commits
5 Commits
init-docs
...
attachment
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
8414ffbcc8 | ||
|
|
5798b50ceb | ||
|
|
e560932211 | ||
|
|
9504e6bac9 | ||
|
|
7086e8e55f |
@@ -1,3 +1,2 @@
|
||||
.claude
|
||||
.github
|
||||
*.test.ts
|
||||
.github
|
||||
108
CLAUDE.md
108
CLAUDE.md
@@ -2,112 +2,72 @@
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Monorepo Structure
|
||||
|
||||
This is a pnpm monorepo containing Svelte MCP (Model Context Protocol) server implementations across multiple packages and applications:
|
||||
|
||||
- **apps/mcp-remote**: SvelteKit web application with MCP server functionality
|
||||
- **packages/mcp-server**: Core MCP server implementation with code analysis tools
|
||||
- **packages/mcp-stdio**: Standalone MCP server CLI with STDIO transport
|
||||
- **packages/mcp-schema**: Shared schema definitions and database utilities
|
||||
|
||||
## Development Commands
|
||||
|
||||
This is a Svelte MCP (Model Context Protocol) server implementation that includes both SvelteKit web interface and MCP server functionality.
|
||||
|
||||
### Setup
|
||||
|
||||
```bash
|
||||
pnpm i
|
||||
cp apps/mcp-remote/.env.example apps/mcp-remote/.env
|
||||
cp .env.example .env
|
||||
# Set the VOYAGE_API_KEY for embeddings support in .env
|
||||
```
|
||||
|
||||
### Starting the mcp-remote app in dev mode
|
||||
|
||||
```bash
|
||||
# Start the SvelteKit development server for mcp-remote (from root)
|
||||
pnpm dev
|
||||
```
|
||||
|
||||
Or navigate to the app directory:
|
||||
|
||||
```bash
|
||||
cd apps/mcp-remote
|
||||
pnpm dev
|
||||
```
|
||||
|
||||
### Common Commands (from root)
|
||||
|
||||
- `pnpm build` - Build all packages and applications
|
||||
- `pnpm check` - Run type checking across all packages
|
||||
- `pnpm lint` - Run prettier check and eslint across all packages
|
||||
- `pnpm format` - Format code with prettier across all packages
|
||||
- `pnpm test` - Run unit tests across all packages
|
||||
- `pnpm test:watch` - Run tests in watch mode
|
||||
|
||||
### mcp-remote App Commands
|
||||
|
||||
Navigate to `apps/mcp-remote/` to run these commands:
|
||||
### Common Commands
|
||||
|
||||
- `pnpm dev` - Start SvelteKit development server
|
||||
- `pnpm build` - Build the SvelteKit application for production
|
||||
- `pnpm build:mcp` - Build the MCP server TypeScript files
|
||||
- `pnpm build` - Build the application for production
|
||||
- `pnpm start` - Run the MCP server (Node.js entry point)
|
||||
- `pnpm check` - Run Svelte type checking
|
||||
- `pnpm check:watch` - Run type checking in watch mode
|
||||
- `pnpm lint` - Run prettier check and eslint
|
||||
- `pnpm format` - Format code with prettier
|
||||
- `pnpm test` - Run unit tests with vitest
|
||||
- `pnpm test:watch` - Run tests in watch mode
|
||||
|
||||
### Database Commands (Drizzle ORM)
|
||||
|
||||
- `pnpm db:push` - Push schema changes to database
|
||||
- `pnpm db:generate` - Generate migration files
|
||||
- `pnpm db:migrate` - Run migrations
|
||||
- `pnpm db:studio` - Open Drizzle Studio
|
||||
- `pnpm inspect` - Start MCP inspector at http://localhost:6274/
|
||||
|
||||
### MCP Inspector Usage
|
||||
|
||||
After running `pnpm inspect`, visit http://localhost:6274/:
|
||||
- Transport type: `Streamable HTTP`
|
||||
- URL: http://localhost:5173/mcp (when dev server is running)
|
||||
|
||||
## Architecture
|
||||
|
||||
### Monorepo Package Structure
|
||||
### MCP Server Implementation
|
||||
|
||||
- **@sveltejs/mcp-remote**: Full SvelteKit application with web interface and MCP server
|
||||
- **@sveltejs/mcp-server**: Core MCP server logic and code analysis engine (private workspace package)
|
||||
- **@sveltejs/mcp**: Standalone CLI MCP server with STDIO transport (publishable)
|
||||
- **@sveltejs/mcp-schema**: Shared database schema and utilities (private workspace package)
|
||||
The core MCP server is implemented in `src/lib/mcp/index.ts` using the `tmcp` library with:
|
||||
|
||||
### mcp-remote App (apps/mcp-remote)
|
||||
- **Transport Layers**: Both HTTP (`HttpTransport`) and STDIO (`StdioTransport`) support
|
||||
- **Schema Validation**: Uses Valibot with `ValibotJsonSchemaAdapter`
|
||||
- **Main Tool**: `svelte-autofixer` - analyzes Svelte code and provides suggestions/fixes
|
||||
|
||||
The main SvelteKit application that provides both web interface and MCP server functionality:
|
||||
### Code Analysis Engine
|
||||
|
||||
- **Entry Point**: `src/index.js` for Node.js MCP server
|
||||
- **SvelteKit Integration**: `src/hooks.server.ts` integrates MCP HTTP transport with SvelteKit requests
|
||||
- **MCP Server**: `src/lib/mcp/index.ts` - HTTP and STDIO transport support
|
||||
- **Database**: SQLite with Drizzle ORM, vector storage for embeddings
|
||||
- **Content Sync**: `src/lib/server/contentSync.ts` and `src/lib/server/contentDb.ts` for content management
|
||||
Located in `src/lib/server/analyze/`:
|
||||
|
||||
### mcp-server Package (packages/mcp-server)
|
||||
- **Parser** (`parse.ts`): Uses `svelte-eslint-parser` and TypeScript parser to analyze Svelte components
|
||||
- **Scope Analysis**: Tracks variables, references, and scopes across the AST
|
||||
- **Rune Detection**: Identifies Svelte 5 runes (`$state`, `$effect`, `$derived`, etc.)
|
||||
|
||||
Core MCP server implementation shared across applications:
|
||||
### Autofixer System
|
||||
|
||||
- **Main Export**: `src/index.ts`
|
||||
- **MCP Implementation**: `src/mcp/index.ts` using `tmcp` library with Valibot schema validation
|
||||
- **Code Analysis**: Svelte component parsing with `svelte-eslint-parser` and TypeScript parser
|
||||
- **Autofixers**: Visitor pattern implementations for code analysis and suggestions
|
||||
- **Tools**: `svelte-autofixer` - analyzes Svelte code and provides suggestions/fixes
|
||||
- **Autofixers** (`src/lib/mcp/autofixers.ts`): Visitor pattern implementations for code analysis
|
||||
- **Walker Utility** (`src/lib/index.ts`): Enhanced AST walking with visitor mixing capabilities
|
||||
- **Current Autofixer**: `assign_in_effect` - detects assignments to `$state` variables inside `$effect` blocks
|
||||
|
||||
### mcp-stdio Package (packages/mcp-stdio)
|
||||
### Database Layer
|
||||
|
||||
Standalone publishable MCP server with STDIO transport:
|
||||
- **ORM**: Drizzle with SQLite backend
|
||||
- **Schema** (`src/lib/server/db/schema.ts`): Vector table for embeddings support
|
||||
- **Utils** (`src/lib/server/db/utils.ts`): Custom float32 array type for vectors
|
||||
|
||||
- **CLI Binary**: `svelte-mcp` command
|
||||
- **Entry Point**: `src/index.ts`
|
||||
- **Transport**: Uses `@tmcp/transport-stdio` for command-line integration
|
||||
### SvelteKit Integration
|
||||
|
||||
### Database Layer (mcp-remote)
|
||||
|
||||
- **ORM**: Drizzle with SQLite backend (`test.db`)
|
||||
- **Schema**: Located in `src/lib/server/db/schema.ts` with vector table for embeddings
|
||||
- **Configuration**: `drizzle.config.ts` in mcp-remote app
|
||||
- **Hooks** (`src/hooks.server.ts`): Integrates MCP HTTP transport with SvelteKit requests
|
||||
- **Routes**: Basic web interface for the MCP server
|
||||
|
||||
## Key Dependencies
|
||||
|
||||
@@ -121,7 +81,7 @@ Standalone publishable MCP server with STDIO transport:
|
||||
|
||||
## Environment Configuration
|
||||
|
||||
For the mcp-remote app (`apps/mcp-remote/.env`):
|
||||
Required environment variables:
|
||||
|
||||
- `DATABASE_URL`: SQLite database path (default: `file:test.db`)
|
||||
- `VOYAGE_API_KEY`: API key for embeddings support (optional)
|
||||
|
||||
@@ -6,7 +6,7 @@ Repo for the official Svelte MCP server.
|
||||
|
||||
```
|
||||
pnpm i
|
||||
cp apps/mcp-remote/.env.example apps/mcp-remote/.env
|
||||
cp .env.example .env
|
||||
pnpm dev
|
||||
```
|
||||
|
||||
|
||||
@@ -1,14 +1,12 @@
|
||||
import { defineConfig } from 'drizzle-kit';
|
||||
|
||||
if (!process.env.DATABASE_URL) throw new Error('DATABASE_URL is not set');
|
||||
if (!process.env.DATABASE_TOKEN) throw new Error('DATABASE_TOKEN is not set');
|
||||
|
||||
export default defineConfig({
|
||||
schema: './src/lib/server/db/schema.ts',
|
||||
dialect: 'turso',
|
||||
dbCredentials: {
|
||||
url: process.env.DATABASE_URL,
|
||||
authToken: process.env.DATABASE_TOKEN || '',
|
||||
},
|
||||
dbCredentials: { url: process.env.DATABASE_URL, authToken: process.env.DATABASE_TOKEN },
|
||||
verbose: true,
|
||||
strict: true,
|
||||
});
|
||||
|
||||
@@ -41,7 +41,6 @@
|
||||
"@eslint/js": "^9.36.0",
|
||||
"@libsql/client": "^0.14.0",
|
||||
"@modelcontextprotocol/inspector": "^0.16.7",
|
||||
"@sveltejs/adapter-node": "^5.3.2",
|
||||
"@sveltejs/adapter-vercel": "^5.6.3",
|
||||
"@sveltejs/kit": "^2.22.0",
|
||||
"@sveltejs/vite-plugin-svelte": "^6.0.0",
|
||||
@@ -65,9 +64,6 @@
|
||||
"dependencies": {
|
||||
"@sveltejs/mcp-schema": "workspace:^",
|
||||
"@sveltejs/mcp-server": "workspace:^",
|
||||
"@tmcp/transport-http": "^0.6.2",
|
||||
"@types/tar-stream": "^3.1.4",
|
||||
"minimatch": "^10.0.3",
|
||||
"tar-stream": "^3.1.7"
|
||||
"@tmcp/transport-http": "^0.6.2"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,138 +0,0 @@
|
||||
import { db } from '$lib/server/db';
|
||||
import { cache } from '$lib/server/db/schema';
|
||||
import { and, eq, sql } from 'drizzle-orm';
|
||||
|
||||
export interface CacheEntry {
|
||||
id: number;
|
||||
cache_key: string;
|
||||
data: Buffer;
|
||||
size_bytes: number;
|
||||
expires_at: Date;
|
||||
created_at: Date;
|
||||
}
|
||||
|
||||
export class CacheDbService {
|
||||
private defaultTTL: number;
|
||||
|
||||
constructor(defaultTTLMinutes: number = 60) {
|
||||
this.defaultTTL = defaultTTLMinutes;
|
||||
}
|
||||
|
||||
async get(key: string): Promise<Buffer | null> {
|
||||
try {
|
||||
const result = await db
|
||||
.select({ data: cache.data })
|
||||
.from(cache)
|
||||
.where(and(eq(cache.cache_key, key), sql`${cache.expires_at} > ${new Date()}`))
|
||||
.limit(1);
|
||||
|
||||
if (result.length === 0) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return result[0].data;
|
||||
} catch (error) {
|
||||
console.error('Error getting cache entry:', error);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
async set(key: string, data: Buffer, ttlMinutes?: number): Promise<void> {
|
||||
const ttl = ttlMinutes || this.defaultTTL;
|
||||
const expires_at = new Date(Date.now() + ttl * 60 * 1000);
|
||||
const now = new Date();
|
||||
|
||||
try {
|
||||
await db
|
||||
.insert(cache)
|
||||
.values({
|
||||
cache_key: key,
|
||||
data,
|
||||
size_bytes: data.length,
|
||||
expires_at,
|
||||
created_at: now,
|
||||
updated_at: now,
|
||||
})
|
||||
.onConflictDoUpdate({
|
||||
target: cache.cache_key,
|
||||
set: {
|
||||
data,
|
||||
size_bytes: data.length,
|
||||
expires_at,
|
||||
updated_at: now,
|
||||
},
|
||||
});
|
||||
} catch (error) {
|
||||
console.error('Error setting cache entry:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async delete(key: string): Promise<boolean> {
|
||||
try {
|
||||
const result = await db.delete(cache).where(eq(cache.cache_key, key));
|
||||
return result.rowsAffected > 0;
|
||||
} catch (error) {
|
||||
console.error('Error deleting cache entry:', error);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
async clear(): Promise<void> {
|
||||
try {
|
||||
await db.delete(cache);
|
||||
} catch (error) {
|
||||
console.error('Error clearing cache:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async deleteExpired(): Promise<number> {
|
||||
try {
|
||||
const result = await db.delete(cache).where(sql`${cache.expires_at} <= ${new Date()}`);
|
||||
return result.rowsAffected;
|
||||
} catch (error) {
|
||||
console.error('Error deleting expired cache entries:', error);
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
|
||||
async getStatus(): Promise<{ count: number; keys: string[]; totalSizeBytes: number }> {
|
||||
try {
|
||||
const result = await db
|
||||
.select({
|
||||
cache_key: cache.cache_key,
|
||||
size_bytes: cache.size_bytes,
|
||||
})
|
||||
.from(cache)
|
||||
.where(sql`${cache.expires_at} > ${new Date()}`)
|
||||
.orderBy(cache.created_at);
|
||||
|
||||
const keys = result.map((row) => row.cache_key);
|
||||
const totalSizeBytes = result.reduce((sum, row) => sum + row.size_bytes, 0);
|
||||
|
||||
return {
|
||||
count: result.length,
|
||||
keys,
|
||||
totalSizeBytes,
|
||||
};
|
||||
} catch (error) {
|
||||
console.error('Error getting cache status:', error);
|
||||
return { count: 0, keys: [], totalSizeBytes: 0 };
|
||||
}
|
||||
}
|
||||
|
||||
async has(key: string): Promise<boolean> {
|
||||
try {
|
||||
const result = await db
|
||||
.select({ exists: sql`1` })
|
||||
.from(cache)
|
||||
.where(and(eq(cache.cache_key, key), sql`${cache.expires_at} > ${new Date()}`))
|
||||
.limit(1);
|
||||
return result.length > 0;
|
||||
} catch (error) {
|
||||
console.error('Error checking cache entry:', error);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,346 +0,0 @@
|
||||
import type { PresetConfig } from '$lib/presets';
|
||||
import { env } from '$env/dynamic/private';
|
||||
import tarStream from 'tar-stream';
|
||||
import { Readable } from 'stream';
|
||||
import { createGunzip } from 'zlib';
|
||||
import { minimatch } from 'minimatch';
|
||||
import { getPresetContent } from './presetCache';
|
||||
import { CacheDbService } from '$lib/cacheDb';
|
||||
import { log, logAlways, logErrorAlways } from '$lib/log';
|
||||
import { cleanTarballPath } from '$lib/utils/pathUtils';
|
||||
|
||||
let cacheService: CacheDbService | null = null;
|
||||
|
||||
function getCacheService(): CacheDbService {
|
||||
if (!cacheService) {
|
||||
cacheService = new CacheDbService();
|
||||
}
|
||||
return cacheService;
|
||||
}
|
||||
|
||||
function sortFilesWithinGroup(files: string[]): string[] {
|
||||
return files.sort((a, b) => {
|
||||
const aPath = a.split('\n')[0].replace('## ', '');
|
||||
const bPath = b.split('\n')[0].replace('## ', '');
|
||||
|
||||
// Check if one path is a parent of the other
|
||||
if (bPath.startsWith(aPath.replace('/index.md', '/'))) return -1;
|
||||
if (aPath.startsWith(bPath.replace('/index.md', '/'))) return 1;
|
||||
|
||||
return aPath.localeCompare(bPath);
|
||||
});
|
||||
}
|
||||
|
||||
export async function fetchRepositoryTarball(owner: string, repo: string): Promise<Buffer> {
|
||||
const cacheKey = `${owner}/${repo}`;
|
||||
const cache = getCacheService();
|
||||
|
||||
const cachedBuffer = await cache.get(cacheKey);
|
||||
if (cachedBuffer) {
|
||||
logAlways(`Using cached tarball for ${cacheKey} from database`);
|
||||
return cachedBuffer;
|
||||
}
|
||||
|
||||
const url = `https://api.github.com/repos/${owner}/${repo}/tarball`;
|
||||
|
||||
logAlways(`Fetching tarball from: ${url}`);
|
||||
|
||||
const response = await fetch(url, {
|
||||
headers: {
|
||||
Authorization: `Bearer ${env.GITHUB_TOKEN}`,
|
||||
Accept: 'application/vnd.github.v3.raw',
|
||||
},
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`Failed to fetch tarball: ${response.statusText}`);
|
||||
}
|
||||
|
||||
if (!response.body) {
|
||||
throw new Error('Response body is null');
|
||||
}
|
||||
|
||||
const chunks: Uint8Array[] = [];
|
||||
const reader = response.body.getReader();
|
||||
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
chunks.push(value);
|
||||
}
|
||||
|
||||
const buffer = Buffer.concat(chunks);
|
||||
|
||||
// Cache the buffer in database with 60 minutes TTL
|
||||
await cache.set(cacheKey, buffer, 60);
|
||||
|
||||
return buffer;
|
||||
}
|
||||
|
||||
export async function processMarkdownFromTarball(
|
||||
tarballBuffer: Buffer,
|
||||
presetConfig: PresetConfig,
|
||||
includePathInfo: boolean,
|
||||
): Promise<string[] | { path: string; content: string }[]> {
|
||||
const { glob, ignore = [], minimize = undefined } = presetConfig;
|
||||
|
||||
// Create a Map to store files for each glob pattern while maintaining order
|
||||
const globResults = new Map<string, unknown[]>();
|
||||
const filePathsByPattern = new Map<string, string[]>();
|
||||
glob.forEach((pattern) => {
|
||||
globResults.set(pattern, []);
|
||||
filePathsByPattern.set(pattern, []);
|
||||
});
|
||||
|
||||
const extractStream = tarStream.extract();
|
||||
|
||||
let processedFiles = 0;
|
||||
let matchedFiles = 0;
|
||||
|
||||
extractStream.on('entry', (header, stream, next) => {
|
||||
processedFiles++;
|
||||
let matched = false;
|
||||
|
||||
for (const pattern of glob) {
|
||||
if (shouldIncludeFile(header.name, pattern, ignore)) {
|
||||
matched = true;
|
||||
matchedFiles++;
|
||||
|
||||
if (header.type === 'file') {
|
||||
let content = '';
|
||||
stream.on('data', (chunk) => (content += chunk.toString()));
|
||||
stream.on('end', () => {
|
||||
// Use the unified path utility to clean tarball paths
|
||||
const cleanPath = cleanTarballPath(header.name);
|
||||
|
||||
const processedContent = minimizeContent(content, minimize);
|
||||
|
||||
if (includePathInfo) {
|
||||
const files = globResults.get(pattern) || [];
|
||||
files.push({
|
||||
path: cleanPath,
|
||||
content: processedContent,
|
||||
});
|
||||
globResults.set(pattern, files);
|
||||
} else {
|
||||
const contentWithHeader = `## ${cleanPath}\n\n${processedContent}`;
|
||||
|
||||
const files = globResults.get(pattern) || [];
|
||||
files.push(contentWithHeader);
|
||||
globResults.set(pattern, files);
|
||||
}
|
||||
|
||||
const paths = filePathsByPattern.get(pattern) || [];
|
||||
paths.push(cleanPath);
|
||||
filePathsByPattern.set(pattern, paths);
|
||||
|
||||
next();
|
||||
});
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (!matched) {
|
||||
stream.resume();
|
||||
next();
|
||||
}
|
||||
});
|
||||
|
||||
const tarballStream = Readable.from(tarballBuffer);
|
||||
const gunzipStream = createGunzip();
|
||||
|
||||
tarballStream.pipe(gunzipStream).pipe(extractStream);
|
||||
|
||||
await new Promise<void>((resolve) => extractStream.on('finish', resolve));
|
||||
|
||||
logAlways(`Total files processed: ${processedFiles}`);
|
||||
logAlways(`Files matching glob: ${matchedFiles}`);
|
||||
log('\nFinal file order:');
|
||||
|
||||
glob.forEach((pattern, index) => {
|
||||
const paths = filePathsByPattern.get(pattern) || [];
|
||||
const sortedPaths = includePathInfo
|
||||
? paths
|
||||
: sortFilesWithinGroup(paths.map((p) => `## ${p}`)).map((p) => p.replace('## ', ''));
|
||||
|
||||
if (sortedPaths.length > 0) {
|
||||
log(`\nGlob pattern ${index + 1}: ${pattern}`);
|
||||
sortedPaths.forEach((path, i) => {
|
||||
log(` ${i + 1}. ${path}`);
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
// Combine results in the order of glob patterns
|
||||
const orderedResults: unknown[] = [];
|
||||
for (const pattern of glob) {
|
||||
const filesForPattern = globResults.get(pattern) || [];
|
||||
if (includePathInfo) {
|
||||
orderedResults.push(...filesForPattern);
|
||||
} else {
|
||||
orderedResults.push(...sortFilesWithinGroup(filesForPattern as string[]));
|
||||
}
|
||||
}
|
||||
|
||||
return orderedResults as string[] | { path: string; content: string }[];
|
||||
}
|
||||
|
||||
function shouldIncludeFile(filename: string, glob: string, ignore: string[] = []): boolean {
|
||||
const shouldIgnore = ignore.some((pattern) => minimatch(filename, pattern));
|
||||
if (shouldIgnore) {
|
||||
logAlways(`❌ Ignored by pattern: ${filename}`);
|
||||
return false;
|
||||
}
|
||||
|
||||
return minimatch(filename, glob);
|
||||
}
|
||||
|
||||
export async function clearRepositoryCache(): Promise<void> {
|
||||
const cache = getCacheService();
|
||||
await cache.clear();
|
||||
logAlways('Repository cache cleared');
|
||||
}
|
||||
|
||||
export async function getRepositoryCacheStatus(): Promise<{
|
||||
size: number;
|
||||
repositories: string[];
|
||||
totalSizeBytes: number;
|
||||
}> {
|
||||
const cache = getCacheService();
|
||||
const status = await cache.getStatus();
|
||||
return {
|
||||
size: status.count,
|
||||
repositories: status.keys,
|
||||
totalSizeBytes: status.totalSizeBytes,
|
||||
};
|
||||
}
|
||||
|
||||
export interface MinimizeOptions {
|
||||
normalizeWhitespace?: boolean;
|
||||
removeLegacy?: boolean;
|
||||
removePlaygroundLinks?: boolean;
|
||||
removePrettierIgnore?: boolean;
|
||||
removeNoteBlocks?: boolean;
|
||||
removeDetailsBlocks?: boolean;
|
||||
removeHtmlComments?: boolean;
|
||||
removeDiffMarkers?: boolean;
|
||||
}
|
||||
|
||||
const defaultOptions: MinimizeOptions = {
|
||||
normalizeWhitespace: false,
|
||||
removeLegacy: false,
|
||||
removePlaygroundLinks: false,
|
||||
removePrettierIgnore: true,
|
||||
removeNoteBlocks: true,
|
||||
removeDetailsBlocks: true,
|
||||
removeHtmlComments: false,
|
||||
removeDiffMarkers: true,
|
||||
};
|
||||
|
||||
function removeQuoteBlocks(content: string, blockType: string): string {
|
||||
return content
|
||||
.split('\n')
|
||||
.reduce((acc: string[], line: string, index: number, lines: string[]) => {
|
||||
// If we find a block (with or without additional text), skip it and all subsequent blockquote lines
|
||||
if (line.trim().startsWith(`> [!${blockType}]`)) {
|
||||
// Skip all subsequent lines that are part of the blockquote
|
||||
let i = index;
|
||||
while (i < lines.length && (lines[i].startsWith('>') || lines[i].trim() === '')) {
|
||||
i++;
|
||||
}
|
||||
// Update the index to skip all these lines
|
||||
index = i - 1;
|
||||
return acc;
|
||||
}
|
||||
|
||||
acc.push(line);
|
||||
return acc;
|
||||
}, [])
|
||||
.join('\n');
|
||||
}
|
||||
|
||||
function removeDiffMarkersFromContent(content: string): string {
|
||||
let inCodeBlock = false;
|
||||
const lines = content.split('\n');
|
||||
const processedLines = lines.map((line) => {
|
||||
// Track if we're entering or leaving a code block
|
||||
// eslint-disable-next-line no-useless-escape
|
||||
if (line.trim().startsWith('\`\`\`')) {
|
||||
inCodeBlock = !inCodeBlock;
|
||||
return line;
|
||||
}
|
||||
|
||||
if (inCodeBlock) {
|
||||
// Handle lines that end with --- or +++ with possible whitespace after
|
||||
// eslint-disable-next-line no-useless-escape
|
||||
line = line.replace(/(\+{3}|\-{3})[\s]*$/g, '');
|
||||
|
||||
// Handle triple markers at start while preserving indentation
|
||||
// This captures the whitespace before the marker and adds it back
|
||||
// eslint-disable-next-line no-useless-escape
|
||||
line = line.replace(/^(\s*)(\+{3}|\-{3})\s*/g, '$1');
|
||||
|
||||
// Handle single + or - markers at start while preserving indentation
|
||||
// eslint-disable-next-line no-useless-escape
|
||||
line = line.replace(/^(\s*)[\+\-](\s)/g, '$1');
|
||||
|
||||
// Handle multi-line diff blocks where --- or +++ might be in the middle of line
|
||||
// eslint-disable-next-line no-useless-escape
|
||||
line = line.replace(/[\s]*(\+{3}|\-{3})[\s]*/g, '');
|
||||
}
|
||||
|
||||
return line;
|
||||
});
|
||||
|
||||
return processedLines.join('\n');
|
||||
}
|
||||
|
||||
export function minimizeContent(content: string, options?: Partial<MinimizeOptions>): string {
|
||||
const settings: MinimizeOptions = options ? { ...defaultOptions, ...options } : defaultOptions;
|
||||
|
||||
let minimized = content;
|
||||
|
||||
minimized = minimized.replace(/NOTE: do not edit this file, it is generated in.*$/gm, '');
|
||||
|
||||
if (settings.removeDiffMarkers) {
|
||||
minimized = removeDiffMarkersFromContent(minimized);
|
||||
}
|
||||
|
||||
if (settings.removeLegacy) {
|
||||
minimized = removeQuoteBlocks(minimized, 'LEGACY');
|
||||
}
|
||||
|
||||
if (settings.removeNoteBlocks) {
|
||||
minimized = removeQuoteBlocks(minimized, 'NOTE');
|
||||
}
|
||||
|
||||
if (settings.removeDetailsBlocks) {
|
||||
minimized = removeQuoteBlocks(minimized, 'DETAILS');
|
||||
}
|
||||
|
||||
if (settings.removePlaygroundLinks) {
|
||||
// Replace playground URLs with /[link] but keep the original link text
|
||||
minimized = minimized.replace(/\[([^\]]+)\]\(\/playground[^)]+\)/g, '[$1](/REMOVED)');
|
||||
}
|
||||
|
||||
if (settings.removePrettierIgnore) {
|
||||
minimized = minimized
|
||||
.split('\n')
|
||||
.filter((line) => line.trim() !== '<!-- prettier-ignore -->')
|
||||
.join('\n');
|
||||
}
|
||||
|
||||
if (settings.removeHtmlComments) {
|
||||
// Replace all HTML comments (including multi-line) with empty string
|
||||
minimized = minimized.replace(/<!--[\s\S]*?-->/g, '');
|
||||
}
|
||||
|
||||
if (settings.normalizeWhitespace) {
|
||||
minimized = minimized.replace(/\s+/g, ' ');
|
||||
}
|
||||
|
||||
minimized = minimized.trim();
|
||||
|
||||
return minimized;
|
||||
}
|
||||
@@ -1,32 +0,0 @@
|
||||
import { dev } from '$app/environment';
|
||||
// eslint-disable-next-line @typescript-eslint/naming-convention, func-style
|
||||
export const log = (...props: unknown[]) => {
|
||||
if (dev) {
|
||||
console.log(...props);
|
||||
}
|
||||
};
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/naming-convention, func-style
|
||||
export const logWarning = (...props: unknown[]) => {
|
||||
if (dev) {
|
||||
console.warn(...props);
|
||||
}
|
||||
};
|
||||
// eslint-disable-next-line @typescript-eslint/naming-convention, func-style
|
||||
export const logError = (...props: unknown[]) => {
|
||||
if (dev) {
|
||||
console.error(...props);
|
||||
}
|
||||
};
|
||||
// eslint-disable-next-line @typescript-eslint/naming-convention, func-style
|
||||
export const logAlways = (...props: unknown[]) => {
|
||||
console.log(...props);
|
||||
};
|
||||
// eslint-disable-next-line @typescript-eslint/naming-convention, func-style
|
||||
export const logWarningAlways = (...props: unknown[]) => {
|
||||
console.warn(...props);
|
||||
};
|
||||
// eslint-disable-next-line @typescript-eslint/naming-convention, func-style
|
||||
export const logErrorAlways = (...props: unknown[]) => {
|
||||
console.error(...props);
|
||||
};
|
||||
@@ -1,213 +0,0 @@
|
||||
import { ContentSyncService } from '$lib/server/contentSync';
|
||||
import { presets } from '$lib/presets';
|
||||
import { log, logAlways, logErrorAlways } from '$lib/log';
|
||||
import { cleanDocumentationPath } from '$lib/utils/pathUtils';
|
||||
import { CacheDbService } from '$lib/cacheDb';
|
||||
|
||||
// Maximum age of cached content in milliseconds (24 hours)
|
||||
export const MAX_CACHE_AGE_MS = 24 * 60 * 60 * 1000;
|
||||
|
||||
let cacheService: CacheDbService | null = null;
|
||||
|
||||
function getCacheService(): CacheDbService {
|
||||
if (!cacheService) {
|
||||
cacheService = new CacheDbService();
|
||||
}
|
||||
return cacheService;
|
||||
}
|
||||
|
||||
export async function getPresetContent(presetKey: string): Promise<string | null> {
|
||||
try {
|
||||
const preset = presets[presetKey];
|
||||
if (!preset) {
|
||||
log(`Preset not found: ${presetKey}`);
|
||||
return null;
|
||||
}
|
||||
|
||||
// Check cache first
|
||||
const cache = getCacheService();
|
||||
const cacheKey = `preset:${presetKey}`;
|
||||
|
||||
try {
|
||||
const cachedData = await cache.get(cacheKey);
|
||||
if (cachedData) {
|
||||
const cachedContent = cachedData.toString('utf8');
|
||||
logAlways(`Using cached content for preset ${presetKey}`);
|
||||
return cachedContent;
|
||||
}
|
||||
} catch (cacheError) {
|
||||
logErrorAlways(`Error reading cache for preset ${presetKey}:`, cacheError);
|
||||
// Continue with normal flow if cache read fails
|
||||
}
|
||||
|
||||
// Try to get files from the content table first
|
||||
let filesWithPaths = await ContentSyncService.getPresetContentFromDb(presetKey);
|
||||
|
||||
// If no content in database, fetch from GitHub and sync
|
||||
if (!filesWithPaths || filesWithPaths.length === 0) {
|
||||
logAlways(`No content in database for preset ${presetKey}, fetching from GitHub...`);
|
||||
|
||||
// Sync the repository first
|
||||
await ContentSyncService.syncRepository();
|
||||
|
||||
// Try again from database
|
||||
filesWithPaths = await ContentSyncService.getPresetContentFromDb(presetKey);
|
||||
|
||||
if (!filesWithPaths || filesWithPaths.length === 0) {
|
||||
log(`Still no content found for preset: ${presetKey} after sync`);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
// Format files with headers and preserve the order from database
|
||||
// The files are already correctly ordered by glob pattern precedence
|
||||
// Use the unified path utility to clean paths
|
||||
const files = filesWithPaths.map((f) => {
|
||||
const cleanPath = cleanDocumentationPath(f.path);
|
||||
return `## ${cleanPath}\n\n${f.content}`;
|
||||
});
|
||||
|
||||
// DO NOT sort - files are already in correct glob pattern order from ContentSyncService
|
||||
const content = files.join('\n\n');
|
||||
|
||||
logAlways(`Generated content for ${presetKey} on-demand (${filesWithPaths.length} files)`);
|
||||
|
||||
// Cache the generated content for 1 hour (60 minutes)
|
||||
try {
|
||||
const contentBuffer = Buffer.from(content, 'utf8');
|
||||
await cache.set(cacheKey, contentBuffer, 60); // 60 minutes TTL
|
||||
logAlways(`Cached content for preset ${presetKey} (expires in 1 hour)`);
|
||||
} catch (cacheError) {
|
||||
logErrorAlways(`Error caching content for preset ${presetKey}:`, cacheError);
|
||||
// Don't fail the request if caching fails
|
||||
}
|
||||
|
||||
return content;
|
||||
} catch (error) {
|
||||
logErrorAlways(`Error generating preset content for ${presetKey}:`, error);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export async function getPresetSizeKb(presetKey: string): Promise<number | null> {
|
||||
try {
|
||||
const content = await getPresetContent(presetKey);
|
||||
if (!content) {
|
||||
return null;
|
||||
}
|
||||
|
||||
const sizeKb = Math.floor(new TextEncoder().encode(content).length / 1024);
|
||||
return sizeKb;
|
||||
} catch (error) {
|
||||
logErrorAlways(`Error calculating preset size for ${presetKey}:`, error);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export async function isPresetStale(presetKey: string): Promise<boolean> {
|
||||
try {
|
||||
// Check if the repository content is stale
|
||||
return await ContentSyncService.isRepositoryContentStale();
|
||||
} catch (error) {
|
||||
logErrorAlways(`Error checking preset staleness for ${presetKey}:`, error);
|
||||
return true; // On error, assume stale
|
||||
}
|
||||
}
|
||||
|
||||
export async function presetExists(presetKey: string): Promise<boolean> {
|
||||
try {
|
||||
const preset = presets[presetKey];
|
||||
if (!preset) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// A preset "exists" if it's defined in presets.ts
|
||||
// The content will be generated on-demand
|
||||
return true;
|
||||
} catch (error) {
|
||||
logErrorAlways(`Error checking preset existence for ${presetKey}:`, error);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export async function getPresetMetadata(presetKey: string): Promise<{
|
||||
size_kb: number;
|
||||
document_count: number;
|
||||
updated_at: Date;
|
||||
is_stale: boolean;
|
||||
} | null> {
|
||||
try {
|
||||
const preset = presets[presetKey];
|
||||
if (!preset) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Try to get files from content table or GitHub
|
||||
const content = await getPresetContent(presetKey);
|
||||
if (!content) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Get the files again to count them (this will use cached data)
|
||||
const filesWithPaths = await ContentSyncService.getPresetContentFromDb(presetKey);
|
||||
const documentCount = filesWithPaths?.length || 0;
|
||||
|
||||
const sizeKb = Math.floor(new TextEncoder().encode(content).length / 1024);
|
||||
const isStale = await isPresetStale(presetKey);
|
||||
|
||||
return {
|
||||
size_kb: sizeKb,
|
||||
document_count: documentCount,
|
||||
updated_at: new Date(), // Since it's generated on-demand, it's always "now"
|
||||
is_stale: isStale,
|
||||
};
|
||||
} catch (error) {
|
||||
logErrorAlways(`Error getting preset metadata for ${presetKey}:`, error);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear the cache for a specific preset
|
||||
*/
|
||||
export async function clearPresetCache(presetKey: string): Promise<boolean> {
|
||||
try {
|
||||
const cache = getCacheService();
|
||||
const cacheKey = `preset:${presetKey}`;
|
||||
const success = await cache.delete(cacheKey);
|
||||
|
||||
if (success) {
|
||||
logAlways(`Cleared cache for preset ${presetKey}`);
|
||||
}
|
||||
|
||||
return success;
|
||||
} catch (error) {
|
||||
logErrorAlways(`Error clearing cache for preset ${presetKey}:`, error);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear cache for all presets
|
||||
*/
|
||||
export async function clearAllPresetCaches(): Promise<number> {
|
||||
try {
|
||||
const cache = getCacheService();
|
||||
const allPresetKeys = Object.keys(presets);
|
||||
let clearedCount = 0;
|
||||
|
||||
for (const presetKey of allPresetKeys) {
|
||||
const cacheKey = `preset:${presetKey}`;
|
||||
const success = await cache.delete(cacheKey);
|
||||
if (success) {
|
||||
clearedCount++;
|
||||
}
|
||||
}
|
||||
|
||||
logAlways(`Cleared cache for ${clearedCount} presets`);
|
||||
return clearedCount;
|
||||
} catch (error) {
|
||||
logErrorAlways(`Error clearing all preset caches:`, error);
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
@@ -1,267 +0,0 @@
|
||||
import type { MinimizeOptions } from './fetchMarkdown';
|
||||
import { SVELTE_5_PROMPT } from '$lib/utils/prompts';
|
||||
|
||||
export type PresetConfig = {
|
||||
title: string;
|
||||
description?: string;
|
||||
glob: string[];
|
||||
ignore?: string[];
|
||||
prompt?: string;
|
||||
minimize?: MinimizeOptions;
|
||||
distilled?: boolean;
|
||||
distilledFilenameBase?: string;
|
||||
};
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/naming-convention
|
||||
export const combinedPresets: Record<string, PresetConfig> = {
|
||||
'svelte-complete-distilled': {
|
||||
title: '🔮 Svelte + SvelteKit (Recommended - LLM Distilled)',
|
||||
description: 'AI-condensed version of the docs focused on code examples and key concepts',
|
||||
glob: [
|
||||
// Svelte
|
||||
'**/apps/svelte.dev/content/docs/svelte/**/*.md',
|
||||
// SvelteKit
|
||||
'**/apps/svelte.dev/content/docs/kit/**/*.md',
|
||||
],
|
||||
minimize: {
|
||||
normalizeWhitespace: false,
|
||||
removeLegacy: true,
|
||||
removePlaygroundLinks: true,
|
||||
removePrettierIgnore: true,
|
||||
removeNoteBlocks: false,
|
||||
removeDetailsBlocks: false,
|
||||
removeHtmlComments: true,
|
||||
removeDiffMarkers: true,
|
||||
},
|
||||
ignore: [
|
||||
// Svelte ignores (same as medium preset)
|
||||
'**/apps/svelte.dev/content/docs/svelte/07-misc/04-custom-elements.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/07-misc/06-v4-migration-guide.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/07-misc/07-v5-migration-guide.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/07-misc/99-faq.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/07-misc/xx-reactivity-indepth.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/98-reference/21-svelte-legacy.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/99-legacy/**/*.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/98-reference/**/*.md',
|
||||
'**/xx-*.md',
|
||||
// SvelteKit ignores (same as medium preset)
|
||||
'**/apps/svelte.dev/content/docs/kit/25-build-and-deploy/*adapter-*.md',
|
||||
'**/apps/svelte.dev/content/docs/kit/25-build-and-deploy/99-writing-adapters.md',
|
||||
'**/apps/svelte.dev/content/docs/kit/30-advanced/70-packaging.md',
|
||||
'**/apps/svelte.dev/content/docs/kit/40-best-practices/05-performance.md',
|
||||
'**/apps/svelte.dev/content/docs/kit/40-best-practices/10-accessibility.md',
|
||||
'**/apps/svelte.dev/content/docs/kit/60-appendix/**/*.md',
|
||||
'**/apps/svelte.dev/content/docs/kit/98-reference/**/*.md',
|
||||
'**/xx-*.md',
|
||||
],
|
||||
prompt: SVELTE_5_PROMPT,
|
||||
distilled: true,
|
||||
distilledFilenameBase: 'svelte-complete-distilled',
|
||||
},
|
||||
'svelte-complete-medium': {
|
||||
title: '⭐️ Svelte + SvelteKit (Medium preset)',
|
||||
description:
|
||||
'Complete Svelte + SvelteKit docs excluding certain advanced sections, legacy, notes and migration docs',
|
||||
glob: [
|
||||
// Svelte
|
||||
'**/apps/svelte.dev/content/docs/svelte/**/*.md',
|
||||
// SvelteKit
|
||||
'**/apps/svelte.dev/content/docs/kit/**/*.md',
|
||||
],
|
||||
ignore: [
|
||||
// Svelte ignores
|
||||
'**/apps/svelte.dev/content/docs/svelte/07-misc/04-custom-elements.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/07-misc/06-v4-migration-guide.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/07-misc/07-v5-migration-guide.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/07-misc/99-faq.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/07-misc/xx-reactivity-indepth.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/98-reference/21-svelte-legacy.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/99-legacy/**/*.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/98-reference/30-runtime-errors.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/98-reference/30-runtime-warnings.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/98-reference/30-compiler-errors.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/98-reference/30-compiler-warnings.md',
|
||||
'**/xx-*.md',
|
||||
// SvelteKit ignores
|
||||
'**/apps/svelte.dev/content/docs/kit/25-build-and-deploy/*adapter-*.md',
|
||||
'**/apps/svelte.dev/content/docs/kit/25-build-and-deploy/99-writing-adapters.md',
|
||||
'**/apps/svelte.dev/content/docs/kit/30-advanced/70-packaging.md',
|
||||
'**/apps/svelte.dev/content/docs/kit/40-best-practices/05-performance.md',
|
||||
'**/apps/svelte.dev/content/docs/kit/40-best-practices/10-accessibility.md', // May the a11y gods have mercy on our souls
|
||||
'**/apps/svelte.dev/content/docs/kit/60-appendix/**/*.md',
|
||||
'**/xx-*.md',
|
||||
],
|
||||
prompt: SVELTE_5_PROMPT,
|
||||
minimize: {
|
||||
removeLegacy: true,
|
||||
removePlaygroundLinks: true,
|
||||
removeNoteBlocks: true,
|
||||
removeDetailsBlocks: true,
|
||||
removeHtmlComments: true,
|
||||
normalizeWhitespace: true,
|
||||
},
|
||||
},
|
||||
'svelte-complete': {
|
||||
title: 'Svelte + SvelteKit (Large preset)',
|
||||
description: 'Complete Svelte + SvelteKit docs excluding legacy, notes and migration docs',
|
||||
glob: [
|
||||
'**/apps/svelte.dev/content/docs/svelte/**/*.md',
|
||||
'**/apps/svelte.dev/content/docs/kit/**/*.md',
|
||||
],
|
||||
ignore: [],
|
||||
prompt: SVELTE_5_PROMPT,
|
||||
minimize: {
|
||||
removeLegacy: true,
|
||||
removePlaygroundLinks: true,
|
||||
removeNoteBlocks: true,
|
||||
removeDetailsBlocks: true,
|
||||
removeHtmlComments: true,
|
||||
normalizeWhitespace: true,
|
||||
},
|
||||
},
|
||||
'svelte-complete-tiny': {
|
||||
title: 'Svelte + SvelteKit (Tiny preset)',
|
||||
description: 'Tutorial content only',
|
||||
glob: [
|
||||
'**/apps/svelte.dev/content/tutorial/**/*.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/02-runes/**/*.md',
|
||||
],
|
||||
ignore: [],
|
||||
prompt: SVELTE_5_PROMPT,
|
||||
minimize: {
|
||||
removeLegacy: true,
|
||||
removePlaygroundLinks: true,
|
||||
removeNoteBlocks: true,
|
||||
removeDetailsBlocks: true,
|
||||
removeHtmlComments: true,
|
||||
normalizeWhitespace: true,
|
||||
},
|
||||
},
|
||||
'svelte-migration': {
|
||||
title: 'Svelte + SvelteKit migration guide',
|
||||
description: 'Only Svelte + SvelteKit docs for migrating ',
|
||||
glob: [
|
||||
// Svelte
|
||||
'**/apps/svelte.dev/content/docs/svelte/07-misc/07-v5-migration-guide.md',
|
||||
// SvelteKit
|
||||
'**/apps/svelte.dev/content/docs/kit/60-appendix/30-migrating-to-sveltekit-2.md',
|
||||
],
|
||||
ignore: [],
|
||||
prompt: SVELTE_5_PROMPT,
|
||||
minimize: {
|
||||
removeLegacy: true,
|
||||
removePlaygroundLinks: true,
|
||||
removeNoteBlocks: true,
|
||||
removeDetailsBlocks: true,
|
||||
removeHtmlComments: true,
|
||||
normalizeWhitespace: true,
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/naming-convention
|
||||
export const sveltePresets: Record<string, PresetConfig> = {
|
||||
svelte: {
|
||||
title: 'Svelte (Full)',
|
||||
description: 'Complete documentation including legacy and reference',
|
||||
glob: ['**/apps/svelte.dev/content/docs/svelte/**/*.md'],
|
||||
ignore: [],
|
||||
prompt: SVELTE_5_PROMPT,
|
||||
minimize: {},
|
||||
},
|
||||
'svelte-medium': {
|
||||
title: 'Svelte (Medium)',
|
||||
description: 'Complete documentation including legacy and reference',
|
||||
glob: ['**/apps/svelte.dev/content/docs/svelte/**/*.md'],
|
||||
ignore: [
|
||||
// Svelte ignores
|
||||
'**/apps/svelte.dev/content/docs/svelte/07-misc/04-custom-elements.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/07-misc/06-v4-migration-guide.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/07-misc/07-v5-migration-guide.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/07-misc/99-faq.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/07-misc/xx-reactivity-indepth.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/98-reference/21-svelte-legacy.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/99-legacy/**/*.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/98-reference/30-runtime-errors.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/98-reference/30-runtime-warnings.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/98-reference/30-compiler-errors.md',
|
||||
'**/apps/svelte.dev/content/docs/svelte/98-reference/30-compiler-warnings.md',
|
||||
],
|
||||
prompt: SVELTE_5_PROMPT,
|
||||
minimize: {
|
||||
removeLegacy: true,
|
||||
removePlaygroundLinks: true,
|
||||
removeNoteBlocks: true,
|
||||
removeDetailsBlocks: true,
|
||||
removeHtmlComments: true,
|
||||
normalizeWhitespace: true,
|
||||
},
|
||||
},
|
||||
};
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/naming-convention
|
||||
export const svelteKitPresets: Record<string, PresetConfig> = {
|
||||
sveltekit: {
|
||||
title: 'SvelteKit (Full)',
|
||||
description: 'Complete documentation including legacy and reference',
|
||||
prompt: SVELTE_5_PROMPT,
|
||||
glob: ['**/apps/svelte.dev/content/docs/kit/**/*.md'],
|
||||
minimize: {},
|
||||
},
|
||||
'sveltekit-medium': {
|
||||
title: 'SvelteKit (Medium)',
|
||||
description: 'Complete documentation including legacy and reference',
|
||||
prompt: SVELTE_5_PROMPT,
|
||||
glob: ['**/apps/svelte.dev/content/docs/kit/**/*.md'],
|
||||
minimize: {
|
||||
removeLegacy: true,
|
||||
removePlaygroundLinks: true,
|
||||
removeNoteBlocks: true,
|
||||
removeDetailsBlocks: true,
|
||||
removeHtmlComments: true,
|
||||
normalizeWhitespace: true,
|
||||
},
|
||||
ignore: [
|
||||
// SvelteKit ignores
|
||||
'**/apps/svelte.dev/content/docs/kit/25-build-and-deploy/*adapter-*.md',
|
||||
'**/apps/svelte.dev/content/docs/kit/25-build-and-deploy/99-writing-adapters.md',
|
||||
'**/apps/svelte.dev/content/docs/kit/30-advanced/70-packaging.md',
|
||||
'**/apps/svelte.dev/content/docs/kit/40-best-practices/05-performance.md',
|
||||
'**/apps/svelte.dev/content/docs/kit/40-best-practices/10-accessibility.md', // May the a11y gods have mercy on our souls
|
||||
'**/apps/svelte.dev/content/docs/kit/60-appendix/**/*.md',
|
||||
'**/xx-*.md',
|
||||
],
|
||||
},
|
||||
};
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/naming-convention
|
||||
export const otherPresets: Record<string, PresetConfig> = {
|
||||
'svelte-cli': {
|
||||
title: 'Svelte CLI - npx sv',
|
||||
glob: ['**/apps/svelte.dev/content/docs/cli/**/*.md'],
|
||||
ignore: [],
|
||||
minimize: {},
|
||||
},
|
||||
};
|
||||
|
||||
export const presets = {
|
||||
...combinedPresets,
|
||||
...sveltePresets,
|
||||
...svelteKitPresets,
|
||||
...otherPresets,
|
||||
};
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/naming-convention
|
||||
export function transformAndSortPresets(presetsObject: Record<string, PresetConfig>) {
|
||||
return Object.entries(presetsObject)
|
||||
.map(([key, value]) => ({
|
||||
key: key.toLowerCase(),
|
||||
...value,
|
||||
}))
|
||||
.sort();
|
||||
}
|
||||
|
||||
export const DEFAULT_REPOSITORY = {
|
||||
owner: 'sveltejs',
|
||||
repo: 'svelte.dev',
|
||||
} as const;
|
||||
@@ -1,383 +0,0 @@
|
||||
import { query } from '$lib/server/db';
|
||||
import type {
|
||||
DbContent,
|
||||
DbContentDistilled,
|
||||
CreateContentInput,
|
||||
ContentFilter,
|
||||
ContentStats,
|
||||
} from '$lib/types/db';
|
||||
import { logAlways, logErrorAlways } from '$lib/log';
|
||||
|
||||
// Type mapping for table names to their corresponding types
|
||||
type TableTypeMap = {
|
||||
content: DbContent;
|
||||
content_distilled: DbContentDistilled;
|
||||
};
|
||||
|
||||
// Union type for valid table names
|
||||
type TableName = keyof TableTypeMap;
|
||||
|
||||
export class ContentDbService {
|
||||
static extractFilename(path: string): string {
|
||||
return path.split('/').pop() || path;
|
||||
}
|
||||
|
||||
static async upsertContent(input: CreateContentInput): Promise<DbContent> {
|
||||
try {
|
||||
const result = await query(
|
||||
`INSERT INTO content (
|
||||
path, filename, content, size_bytes, metadata
|
||||
) VALUES ($1, $2, $3, $4, $5)
|
||||
ON CONFLICT (path) DO UPDATE SET
|
||||
content = EXCLUDED.content,
|
||||
size_bytes = EXCLUDED.size_bytes,
|
||||
metadata = EXCLUDED.metadata,
|
||||
updated_at = CURRENT_TIMESTAMP
|
||||
RETURNING *`,
|
||||
[
|
||||
input.path,
|
||||
input.filename,
|
||||
input.content,
|
||||
input.size_bytes,
|
||||
input.metadata ? JSON.stringify(input.metadata) : '{}',
|
||||
],
|
||||
);
|
||||
|
||||
logAlways(`Upserted content for ${input.path}`);
|
||||
return result.rows[0] as DbContent;
|
||||
} catch (error) {
|
||||
logErrorAlways(`Failed to upsert content for ${input.path}:`, error);
|
||||
throw new Error(
|
||||
`Failed to upsert content: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
static async getContentByPath(path: string): Promise<DbContent | null> {
|
||||
try {
|
||||
const result = await query('SELECT * FROM content WHERE path = $1', [path]);
|
||||
return result.rows.length > 0 ? (result.rows[0] as DbContent) : null;
|
||||
} catch (error) {
|
||||
logErrorAlways(`Failed to get content ${path}:`, error);
|
||||
throw new Error(
|
||||
`Failed to get content: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
static async getAllContent(): Promise<DbContent[]> {
|
||||
try {
|
||||
const result = await query('SELECT * FROM content ORDER BY path');
|
||||
return result.rows as DbContent[];
|
||||
} catch (error) {
|
||||
logErrorAlways('Failed to get all content:', error);
|
||||
throw new Error(
|
||||
`Failed to get content: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Generic search method that works with both content and content_distilled tables
|
||||
*/
|
||||
static async searchContent<T extends TableName>(
|
||||
searchQuery: string,
|
||||
tableName: T,
|
||||
pathPattern: string = 'apps/svelte.dev/content/docs/%',
|
||||
): Promise<TableTypeMap[T] | null> {
|
||||
try {
|
||||
const lowerQuery = searchQuery.toLowerCase();
|
||||
|
||||
// Build table-specific WHERE clauses
|
||||
let baseWhereClause = '';
|
||||
let params: (string | number)[] = [];
|
||||
let paramIndex = 1;
|
||||
|
||||
if (tableName === 'content') {
|
||||
// For content table, include path filter
|
||||
baseWhereClause = `WHERE path LIKE $${paramIndex}`;
|
||||
params = [pathPattern];
|
||||
paramIndex = 2;
|
||||
} else {
|
||||
// For content_distilled table, no additional filters needed
|
||||
baseWhereClause = '';
|
||||
paramIndex = 1;
|
||||
}
|
||||
|
||||
// First, try exact title match using JSON operators
|
||||
const exactTitleQueryStr = `
|
||||
SELECT * FROM ${tableName}
|
||||
${baseWhereClause}${baseWhereClause ? ' AND' : 'WHERE'} LOWER(metadata->>'title') = $${paramIndex}
|
||||
LIMIT 1
|
||||
`;
|
||||
|
||||
const exactTitleParams = [...params, lowerQuery];
|
||||
const exactTitleResult = await query(exactTitleQueryStr, exactTitleParams);
|
||||
|
||||
if (exactTitleResult.rows.length > 0) {
|
||||
return exactTitleResult.rows[0] as TableTypeMap[T];
|
||||
}
|
||||
|
||||
// Then try partial title match
|
||||
const partialTitleQueryStr = `
|
||||
SELECT * FROM ${tableName}
|
||||
${baseWhereClause}${baseWhereClause ? ' AND' : 'WHERE'} LOWER(metadata->>'title') LIKE $${paramIndex}
|
||||
LIMIT 1
|
||||
`;
|
||||
|
||||
const partialTitleParams = [...params, `%${lowerQuery}%`];
|
||||
const partialTitleResult = await query(partialTitleQueryStr, partialTitleParams);
|
||||
|
||||
if (partialTitleResult.rows.length > 0) {
|
||||
return partialTitleResult.rows[0] as TableTypeMap[T];
|
||||
}
|
||||
|
||||
// Finally try path match for backward compatibility
|
||||
const pathMatchQueryStr = `
|
||||
SELECT * FROM ${tableName}
|
||||
${baseWhereClause}${baseWhereClause ? ' AND' : 'WHERE'} LOWER(path) LIKE $${paramIndex}
|
||||
LIMIT 1
|
||||
`;
|
||||
|
||||
const pathMatchParams = [...params, `%${lowerQuery}%`];
|
||||
const pathMatchResult = await query(pathMatchQueryStr, pathMatchParams);
|
||||
|
||||
return pathMatchResult.rows.length > 0 ? (pathMatchResult.rows[0] as TableTypeMap[T]) : null;
|
||||
} catch (error) {
|
||||
logErrorAlways(`Failed to search ${tableName} for "${searchQuery}":`, error);
|
||||
throw new Error(
|
||||
`Failed to search ${tableName}: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
static async searchAllContent(
|
||||
searchQuery: string,
|
||||
pathPattern: string = 'apps/svelte.dev/content/docs/%',
|
||||
limit: number = 50,
|
||||
): Promise<DbContent[]> {
|
||||
try {
|
||||
const lowerQuery = searchQuery.toLowerCase();
|
||||
|
||||
// Combine all search types into one query with UNION
|
||||
const combinedQueryStr = `
|
||||
-- Exact title matches first
|
||||
(SELECT * FROM content
|
||||
WHERE path LIKE $1
|
||||
AND LOWER(metadata->>'title') = $2
|
||||
ORDER BY path
|
||||
LIMIT $3)
|
||||
|
||||
UNION
|
||||
|
||||
-- Then partial title matches
|
||||
(SELECT * FROM content
|
||||
WHERE path LIKE $1
|
||||
AND LOWER(metadata->>'title') LIKE $4
|
||||
AND LOWER(metadata->>'title') != $2
|
||||
ORDER BY path
|
||||
LIMIT $3)
|
||||
|
||||
UNION
|
||||
|
||||
-- Finally path matches
|
||||
(SELECT * FROM content
|
||||
WHERE path LIKE $1
|
||||
AND LOWER(path) LIKE $4
|
||||
AND (metadata->>'title' IS NULL OR LOWER(metadata->>'title') NOT LIKE $4)
|
||||
ORDER BY path
|
||||
LIMIT $3)
|
||||
|
||||
ORDER BY path
|
||||
LIMIT $3
|
||||
`;
|
||||
|
||||
const params = [pathPattern, lowerQuery, limit, `%${lowerQuery}%`];
|
||||
|
||||
const result = await query(combinedQueryStr, params);
|
||||
|
||||
return result.rows as DbContent[];
|
||||
} catch (error) {
|
||||
logErrorAlways(`Failed to search all content for "${searchQuery}":`, error);
|
||||
throw new Error(
|
||||
`Failed to search content: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
static async getDocumentationSections(
|
||||
pathPattern: string = 'apps/svelte.dev/content/docs/%',
|
||||
minContentLength: number = 100,
|
||||
): Promise<Array<{ path: string; metadata: Record<string, unknown>; content: string }>> {
|
||||
try {
|
||||
const sectionsQueryStr = `
|
||||
SELECT path, metadata, content
|
||||
FROM content
|
||||
WHERE path LIKE $1
|
||||
AND LENGTH(content) >= $2
|
||||
ORDER BY path
|
||||
`;
|
||||
|
||||
const params = [pathPattern, minContentLength];
|
||||
|
||||
const result = await query(sectionsQueryStr, params);
|
||||
|
||||
return result.rows.map((row) => ({
|
||||
path: row.path,
|
||||
metadata: row.metadata,
|
||||
content: row.content,
|
||||
}));
|
||||
} catch (error) {
|
||||
logErrorAlways('Failed to get documentation sections:', error);
|
||||
throw new Error(
|
||||
`Failed to get sections: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
static async getFilteredContent(
|
||||
pathPattern: string = 'apps/svelte.dev/content/docs/%',
|
||||
minContentLength: number = 200,
|
||||
): Promise<DbContent[]> {
|
||||
try {
|
||||
const filterQueryStr = `
|
||||
SELECT *
|
||||
FROM content
|
||||
WHERE path LIKE $1
|
||||
AND LENGTH(content) >= $2
|
||||
ORDER BY path
|
||||
`;
|
||||
|
||||
const params = [pathPattern, minContentLength];
|
||||
|
||||
const result = await query(filterQueryStr, params);
|
||||
return result.rows as DbContent[];
|
||||
} catch (error) {
|
||||
logErrorAlways('Failed to get filtered content:', error);
|
||||
throw new Error(
|
||||
`Failed to get filtered content: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
static async getContentStats(): Promise<ContentStats> {
|
||||
try {
|
||||
const totalResult = await query(
|
||||
`SELECT
|
||||
COUNT(*) as total_files,
|
||||
COALESCE(SUM(size_bytes), 0) as total_size_bytes,
|
||||
MAX(updated_at) as last_updated
|
||||
FROM content`,
|
||||
);
|
||||
|
||||
return {
|
||||
total_files: parseInt(totalResult.rows[0].total_files),
|
||||
total_size_bytes: parseInt(totalResult.rows[0].total_size_bytes),
|
||||
last_updated: totalResult.rows[0].last_updated,
|
||||
};
|
||||
} catch (error) {
|
||||
logErrorAlways('Failed to get content stats:', error);
|
||||
throw new Error(
|
||||
`Failed to get stats: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
static async deleteContent(path: string): Promise<boolean> {
|
||||
try {
|
||||
const result = await query('DELETE FROM content WHERE path = $1', [path]);
|
||||
return (result.rowCount ?? 0) > 0;
|
||||
} catch (error) {
|
||||
logErrorAlways(`Failed to delete content ${path}:`, error);
|
||||
throw new Error(
|
||||
`Failed to delete content: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
static async deleteAllContent(): Promise<number> {
|
||||
try {
|
||||
const result = await query('DELETE FROM content');
|
||||
return result.rowCount ?? 0;
|
||||
} catch (error) {
|
||||
logErrorAlways('Failed to delete all content:', error);
|
||||
throw new Error(
|
||||
`Failed to delete content: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
static async hasContentChanged(path: string, newContent: string): Promise<boolean> {
|
||||
try {
|
||||
const existing = await ContentDbService.getContentByPath(path);
|
||||
if (!existing) return true;
|
||||
|
||||
return existing.content !== newContent;
|
||||
} catch (error) {
|
||||
logErrorAlways(`Failed to check content change for ${path}:`, error);
|
||||
return true; // Assume changed on error
|
||||
}
|
||||
}
|
||||
|
||||
static async batchUpsertContent(contents: CreateContentInput[]): Promise<DbContent[]> {
|
||||
try {
|
||||
const results: DbContent[] = [];
|
||||
|
||||
// Process in chunks to avoid overwhelming the database
|
||||
const chunkSize = 200;
|
||||
for (let i = 0; i < contents.length; i += chunkSize) {
|
||||
const chunk = contents.slice(i, i + chunkSize);
|
||||
|
||||
const chunkResults = await Promise.all(
|
||||
chunk.map((content) => ContentDbService.upsertContent(content)),
|
||||
);
|
||||
|
||||
results.push(...chunkResults);
|
||||
}
|
||||
|
||||
logAlways(`Batch upserted ${results.length} content items`);
|
||||
return results;
|
||||
} catch (error) {
|
||||
logErrorAlways('Failed to batch upsert content:', error);
|
||||
throw new Error(
|
||||
`Failed to batch upsert: ${error instanceof Error ? error.message : String(error)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
static extractFrontmatter(content: string): Record<string, unknown> {
|
||||
const metadata: Record<string, unknown> = {};
|
||||
|
||||
if (!content.startsWith('---\n')) {
|
||||
return metadata;
|
||||
}
|
||||
|
||||
const endIndex = content.indexOf('\n---\n', 4);
|
||||
if (endIndex === -1) {
|
||||
return metadata;
|
||||
}
|
||||
|
||||
const frontmatter = content.substring(4, endIndex);
|
||||
const lines = frontmatter.split('\n');
|
||||
|
||||
for (const line of lines) {
|
||||
const colonIndex = line.indexOf(':');
|
||||
if (colonIndex > 0) {
|
||||
const key = line.substring(0, colonIndex).trim();
|
||||
const value = line.substring(colonIndex + 1).trim();
|
||||
|
||||
// Remove quotes if present
|
||||
const cleanValue = value.replace(/^["'](.*)["']$/, '$1');
|
||||
|
||||
// Try to parse as JSON for nested structures
|
||||
try {
|
||||
metadata[key] = JSON.parse(cleanValue);
|
||||
} catch {
|
||||
metadata[key] = cleanValue;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return metadata;
|
||||
}
|
||||
}
|
||||
@@ -1,271 +0,0 @@
|
||||
import {
|
||||
fetchRepositoryTarball,
|
||||
processMarkdownFromTarball,
|
||||
minimizeContent,
|
||||
} from '$lib/fetchMarkdown';
|
||||
import { ContentDbService } from '$lib/server/contentDb';
|
||||
import type { CreateContentInput } from '$lib/types/db';
|
||||
import { presets, DEFAULT_REPOSITORY } from '$lib/presets';
|
||||
import { logAlways, logErrorAlways, log } from '$lib/log';
|
||||
|
||||
function sortFilesWithinGroup(
|
||||
files: Array<{ path: string; content: string }>,
|
||||
): Array<{ path: string; content: string }> {
|
||||
return files.sort((a, b) => {
|
||||
const aPath = a.path;
|
||||
const bPath = b.path;
|
||||
|
||||
// Check if one path is a parent of the other
|
||||
if (bPath.startsWith(aPath.replace('/index.md', '/'))) return -1;
|
||||
if (aPath.startsWith(bPath.replace('/index.md', '/'))) return 1;
|
||||
|
||||
return aPath.localeCompare(bPath);
|
||||
});
|
||||
}
|
||||
|
||||
export class ContentSyncService {
|
||||
static readonly MAX_CONTENT_AGE_MS = 24 * 60 * 60 * 1000;
|
||||
|
||||
static async syncRepository(
|
||||
options: {
|
||||
returnStats?: boolean;
|
||||
} = {},
|
||||
): Promise<{
|
||||
success: boolean;
|
||||
stats: {
|
||||
total_files: number;
|
||||
total_size_bytes: number;
|
||||
last_updated: Date;
|
||||
};
|
||||
sync_details: {
|
||||
upserted_files: number;
|
||||
deleted_files: number;
|
||||
unchanged_files: number;
|
||||
};
|
||||
timestamp: string;
|
||||
}> {
|
||||
const { returnStats = true } = options;
|
||||
const { owner, repo: repoName } = DEFAULT_REPOSITORY;
|
||||
|
||||
logAlways(`Starting sync for repository: ${owner}/${repoName}`);
|
||||
|
||||
let upsertedFiles = 0;
|
||||
let deletedFiles = 0;
|
||||
let unchangedFiles = 0;
|
||||
|
||||
try {
|
||||
logAlways(`Step 1: Syncing repository ${owner}/${repoName}`);
|
||||
|
||||
const tarballBuffer = await fetchRepositoryTarball(owner, repoName);
|
||||
|
||||
const filesWithPaths = (await processMarkdownFromTarball(
|
||||
tarballBuffer,
|
||||
{
|
||||
glob: ['**/*.md', '**/*.mdx'],
|
||||
ignore: [],
|
||||
title: `Sync ${owner}/${repoName}`,
|
||||
distilled: false,
|
||||
},
|
||||
true,
|
||||
)) as Array<{
|
||||
path: string;
|
||||
content: string;
|
||||
}>;
|
||||
|
||||
logAlways(`Found ${filesWithPaths.length} markdown files in ${owner}/${repoName}`);
|
||||
|
||||
const existingFiles = await ContentDbService.getAllContent();
|
||||
const existingPaths = new Set(existingFiles.map((file) => file.path));
|
||||
|
||||
const foundPaths = new Set(filesWithPaths.map((file) => file.path));
|
||||
|
||||
const contentInputs: CreateContentInput[] = [];
|
||||
|
||||
for (const file of filesWithPaths) {
|
||||
const filename = ContentDbService.extractFilename(file.path);
|
||||
const sizeBytes = new TextEncoder().encode(file.content).length;
|
||||
|
||||
const metadata = ContentDbService.extractFrontmatter(file.content);
|
||||
|
||||
const hasChanged = await ContentDbService.hasContentChanged(file.path, file.content);
|
||||
|
||||
if (hasChanged) {
|
||||
contentInputs.push({
|
||||
path: file.path,
|
||||
filename,
|
||||
content: file.content,
|
||||
size_bytes: sizeBytes,
|
||||
metadata,
|
||||
});
|
||||
} else {
|
||||
unchangedFiles++;
|
||||
}
|
||||
}
|
||||
|
||||
if (contentInputs.length > 0) {
|
||||
logAlways(`Upserting ${contentInputs.length} changed files`);
|
||||
await ContentDbService.batchUpsertContent(contentInputs);
|
||||
upsertedFiles = contentInputs.length;
|
||||
} else {
|
||||
logAlways(`No file content changes detected`);
|
||||
}
|
||||
|
||||
// Handle deletions - find files in DB that are no longer in the repository
|
||||
const deletedPaths = Array.from(existingPaths).filter((path) => !foundPaths.has(path));
|
||||
|
||||
if (deletedPaths.length > 0) {
|
||||
logAlways(`Deleting ${deletedPaths.length} files that no longer exist`);
|
||||
|
||||
for (const deletedPath of deletedPaths) {
|
||||
logAlways(` Deleting: ${deletedPath}`);
|
||||
await ContentDbService.deleteContent(deletedPath);
|
||||
}
|
||||
deletedFiles = deletedPaths.length;
|
||||
} else {
|
||||
logAlways(`No deleted files detected`);
|
||||
}
|
||||
|
||||
let stats;
|
||||
if (returnStats) {
|
||||
logAlways(`Step 2: Collecting final statistics`);
|
||||
stats = await ContentSyncService.getContentStats();
|
||||
} else {
|
||||
logAlways(`Step 2: Skipping stats collection (returnStats = false)`);
|
||||
// Return minimal stats structure
|
||||
stats = {
|
||||
total_files: 0,
|
||||
total_size_bytes: 0,
|
||||
last_updated: new Date(),
|
||||
};
|
||||
}
|
||||
|
||||
logAlways(
|
||||
`Sync completed successfully: ${upsertedFiles} upserted, ${deletedFiles} deleted, ${unchangedFiles} unchanged`,
|
||||
);
|
||||
|
||||
return {
|
||||
success: true,
|
||||
stats,
|
||||
sync_details: {
|
||||
upserted_files: upsertedFiles,
|
||||
deleted_files: deletedFiles,
|
||||
unchanged_files: unchangedFiles,
|
||||
},
|
||||
timestamp: new Date().toISOString(),
|
||||
};
|
||||
} catch (error) {
|
||||
logErrorAlways(`Failed to sync repository ${owner}/${repoName}:`, error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
static async isRepositoryContentStale(): Promise<boolean> {
|
||||
try {
|
||||
const stats = await ContentDbService.getContentStats();
|
||||
|
||||
if (stats.total_files === 0) {
|
||||
return true; // No content, consider stale
|
||||
}
|
||||
|
||||
const lastUpdated = new Date(stats.last_updated);
|
||||
const contentAge = Date.now() - lastUpdated.getTime();
|
||||
|
||||
const isStale = contentAge > ContentSyncService.MAX_CONTENT_AGE_MS;
|
||||
|
||||
if (isStale) {
|
||||
logAlways(
|
||||
`Repository content is stale (age: ${Math.floor(contentAge / 1000 / 60)} minutes)`,
|
||||
);
|
||||
}
|
||||
|
||||
return isStale;
|
||||
} catch (error) {
|
||||
logErrorAlways(`Error checking repository staleness:`, error);
|
||||
return true; // On error, assume stale
|
||||
}
|
||||
}
|
||||
|
||||
static async getPresetContentFromDb(
|
||||
presetKey: string,
|
||||
): Promise<Array<{ path: string; content: string }> | null> {
|
||||
const preset = presets[presetKey];
|
||||
if (!preset) {
|
||||
return null;
|
||||
}
|
||||
|
||||
try {
|
||||
const allContent = await ContentDbService.getAllContent();
|
||||
|
||||
if (allContent.length === 0) {
|
||||
return null;
|
||||
}
|
||||
|
||||
log(`Checking ${allContent.length} files against glob patterns for preset ${presetKey}`);
|
||||
log(`Glob patterns: ${JSON.stringify(preset.glob)}`);
|
||||
log(`Ignore patterns: ${JSON.stringify(preset.ignore || [])}`);
|
||||
|
||||
const { minimatch } = await import('minimatch');
|
||||
|
||||
const orderedResults: Array<{ path: string; content: string }> = [];
|
||||
|
||||
// Process one glob pattern at a time
|
||||
for (const pattern of preset.glob) {
|
||||
log(`\nProcessing glob pattern: ${pattern}`);
|
||||
|
||||
const matchingFiles: Array<{ path: string; content: string }> = [];
|
||||
|
||||
for (const dbContent of allContent) {
|
||||
const shouldIgnore = preset.ignore?.some((ignorePattern) => {
|
||||
const matches = minimatch(dbContent.path, ignorePattern);
|
||||
if (matches) {
|
||||
log(` File ${dbContent.path} ignored by pattern: ${ignorePattern}`);
|
||||
}
|
||||
return matches;
|
||||
});
|
||||
if (shouldIgnore) continue;
|
||||
|
||||
if (minimatch(dbContent.path, pattern)) {
|
||||
log(` File ${dbContent.path} matched`);
|
||||
|
||||
let processedContent = dbContent.content;
|
||||
if (preset.minimize && Object.keys(preset.minimize).length > 0) {
|
||||
processedContent = minimizeContent(dbContent.content, preset.minimize);
|
||||
}
|
||||
|
||||
matchingFiles.push({
|
||||
path: dbContent.path,
|
||||
content: processedContent,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const sortedFiles = sortFilesWithinGroup(matchingFiles);
|
||||
|
||||
log(` Found ${sortedFiles.length} files for pattern: ${pattern}`);
|
||||
sortedFiles.forEach((file, i) => {
|
||||
log(` ${i + 1}. ${file.path}`);
|
||||
});
|
||||
|
||||
orderedResults.push(...sortedFiles);
|
||||
}
|
||||
|
||||
logAlways(
|
||||
`Found ${orderedResults.length} files matching preset ${presetKey} from database in natural glob order`,
|
||||
);
|
||||
|
||||
log('\nFinal file order:');
|
||||
orderedResults.forEach((file, i) => {
|
||||
log(` ${i + 1}. ${file.path}`);
|
||||
});
|
||||
|
||||
return orderedResults;
|
||||
} catch (error) {
|
||||
logErrorAlways(`Failed to get preset content from database for ${presetKey}:`, error);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
static async getContentStats() {
|
||||
return ContentDbService.getContentStats();
|
||||
}
|
||||
}
|
||||
@@ -1,112 +0,0 @@
|
||||
export interface QueryConfig {
|
||||
debug?: boolean;
|
||||
}
|
||||
|
||||
// Enum for distillable preset names
|
||||
export enum DistillablePreset {
|
||||
SVELTE_DISTILLED = 'svelte-distilled',
|
||||
SVELTEKIT_DISTILLED = 'sveltekit-distilled',
|
||||
SVELTE_COMPLETE_DISTILLED = 'svelte-complete-distilled',
|
||||
}
|
||||
|
||||
// Database table types
|
||||
|
||||
export interface DbDistillation {
|
||||
id: number;
|
||||
preset_name: DistillablePreset;
|
||||
version: string; // 'latest' or '2024-01-15'
|
||||
content: string;
|
||||
size_kb: number;
|
||||
document_count: number;
|
||||
distillation_job_id: number | null;
|
||||
created_at: Date;
|
||||
}
|
||||
|
||||
export interface DbDistillationJob {
|
||||
id: number;
|
||||
preset_name: string;
|
||||
batch_id: string | null;
|
||||
status: 'pending' | 'processing' | 'completed' | 'failed';
|
||||
model_used: string;
|
||||
total_files: number;
|
||||
processed_files: number;
|
||||
successful_files: number;
|
||||
minimize_applied: boolean;
|
||||
total_input_tokens: number;
|
||||
total_output_tokens: number;
|
||||
started_at: Date | null;
|
||||
completed_at: Date | null;
|
||||
error_message: string | null;
|
||||
metadata: Record<string, unknown>; // JSONB
|
||||
created_at: Date;
|
||||
updated_at: Date;
|
||||
}
|
||||
|
||||
export interface DbContent {
|
||||
id: number;
|
||||
path: string;
|
||||
filename: string;
|
||||
content: string;
|
||||
size_bytes: number;
|
||||
metadata: Record<string, unknown>;
|
||||
created_at: Date;
|
||||
updated_at: Date;
|
||||
}
|
||||
|
||||
export interface DbContentDistilled {
|
||||
id: number;
|
||||
path: string;
|
||||
filename: string;
|
||||
content: string;
|
||||
size_bytes: number;
|
||||
metadata: Record<string, unknown>;
|
||||
created_at: Date;
|
||||
updated_at: Date;
|
||||
}
|
||||
|
||||
// Input types for creating/updating records
|
||||
|
||||
export interface CreateDistillationInput {
|
||||
preset_name: DistillablePreset;
|
||||
version: string;
|
||||
content: string;
|
||||
size_kb: number;
|
||||
document_count: number;
|
||||
distillation_job_id?: number;
|
||||
}
|
||||
|
||||
export interface CreateDistillationJobInput {
|
||||
preset_name: string;
|
||||
batch_id?: string;
|
||||
status: 'pending' | 'processing' | 'completed' | 'failed';
|
||||
model_used: string;
|
||||
total_files: number;
|
||||
minimize_applied?: boolean;
|
||||
metadata?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
export interface CreateContentInput {
|
||||
path: string;
|
||||
filename: string;
|
||||
content: string;
|
||||
size_bytes: number;
|
||||
metadata?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
export interface CreateContentDistilledInput {
|
||||
path: string;
|
||||
filename: string;
|
||||
content: string;
|
||||
size_bytes: number;
|
||||
metadata?: Record<string, unknown>;
|
||||
}
|
||||
|
||||
export interface ContentFilter {
|
||||
path_pattern?: string; // For glob pattern matching
|
||||
}
|
||||
|
||||
export interface ContentStats {
|
||||
total_files: number;
|
||||
total_size_bytes: number;
|
||||
last_updated: Date;
|
||||
}
|
||||
@@ -1,335 +0,0 @@
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import {
|
||||
cleanDocumentationPath,
|
||||
cleanTarballPath,
|
||||
extractTitleFromPath,
|
||||
removeFrontmatter,
|
||||
} from './pathUtils.js';
|
||||
|
||||
describe('pathUtils', () => {
|
||||
describe('cleanDocumentationPath', () => {
|
||||
it('should remove apps/svelte.dev/content/ prefix', () => {
|
||||
const input = 'apps/svelte.dev/content/docs/svelte/01-introduction.md';
|
||||
const expected = 'docs/svelte/01-introduction.md';
|
||||
expect(cleanDocumentationPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle paths without the prefix', () => {
|
||||
const input = 'docs/svelte/01-introduction.md';
|
||||
const expected = 'docs/svelte/01-introduction.md';
|
||||
expect(cleanDocumentationPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle empty string', () => {
|
||||
const input = '';
|
||||
const expected = '';
|
||||
expect(cleanDocumentationPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle partial prefix matches', () => {
|
||||
const input = 'apps/svelte.dev/content-extra/docs/svelte/01-introduction.md';
|
||||
const expected = 'apps/svelte.dev/content-extra/docs/svelte/01-introduction.md';
|
||||
expect(cleanDocumentationPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle paths with similar but different prefixes', () => {
|
||||
const input = 'apps/svelte.dev/contents/docs/svelte/01-introduction.md';
|
||||
const expected = 'apps/svelte.dev/contents/docs/svelte/01-introduction.md';
|
||||
expect(cleanDocumentationPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle SvelteKit documentation paths', () => {
|
||||
const input = 'apps/svelte.dev/content/docs/kit/01-routing.md';
|
||||
const expected = 'docs/kit/01-routing.md';
|
||||
expect(cleanDocumentationPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle tutorial paths', () => {
|
||||
const input = 'apps/svelte.dev/content/tutorial/01-introduction/01-hello-world.md';
|
||||
const expected = 'tutorial/01-introduction/01-hello-world.md';
|
||||
expect(cleanDocumentationPath(input)).toBe(expected);
|
||||
});
|
||||
});
|
||||
|
||||
describe('cleanTarballPath', () => {
|
||||
it('should remove the first segment from tarball paths', () => {
|
||||
const input = 'svelte.dev-main/apps/svelte.dev/content/docs/svelte/01-introduction.md';
|
||||
const expected = 'apps/svelte.dev/content/docs/svelte/01-introduction.md';
|
||||
expect(cleanTarballPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle paths with different repo prefixes', () => {
|
||||
const input = 'svelte-12345/apps/svelte.dev/content/docs/kit/01-routing.md';
|
||||
const expected = 'apps/svelte.dev/content/docs/kit/01-routing.md';
|
||||
expect(cleanTarballPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle single segment paths', () => {
|
||||
const input = 'single-segment';
|
||||
const expected = '';
|
||||
expect(cleanTarballPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle empty string', () => {
|
||||
const input = '';
|
||||
const expected = '';
|
||||
expect(cleanTarballPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle paths with no segments', () => {
|
||||
const input = 'just-filename.md';
|
||||
const expected = '';
|
||||
expect(cleanTarballPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle complex nested paths', () => {
|
||||
const input = 'repo-name/very/deep/nested/path/to/file.md';
|
||||
const expected = 'very/deep/nested/path/to/file.md';
|
||||
expect(cleanTarballPath(input)).toBe(expected);
|
||||
});
|
||||
});
|
||||
|
||||
describe('extractTitleFromPath', () => {
|
||||
it('should extract filename and remove .md extension and numbered prefix', () => {
|
||||
const input = 'docs/svelte/01-introduction.md';
|
||||
const expected = 'introduction';
|
||||
expect(extractTitleFromPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should remove numbered prefixes', () => {
|
||||
const input = 'docs/svelte/01-introduction.md';
|
||||
const expected = 'introduction';
|
||||
expect(extractTitleFromPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle files without numbered prefixes', () => {
|
||||
const input = 'docs/svelte/reactivity.md';
|
||||
const expected = 'reactivity';
|
||||
expect(extractTitleFromPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle files without .md extension', () => {
|
||||
const input = 'docs/svelte/01-introduction';
|
||||
const expected = 'introduction';
|
||||
expect(extractTitleFromPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle complex numbered prefixes', () => {
|
||||
const input = 'docs/svelte/99-advanced-topics.md';
|
||||
const expected = 'advanced-topics';
|
||||
expect(extractTitleFromPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle files with multiple numbered prefixes', () => {
|
||||
const input = 'docs/svelte/01-02-nested-numbering.md';
|
||||
const expected = '02-nested-numbering';
|
||||
expect(extractTitleFromPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle just a filename', () => {
|
||||
const input = '01-introduction.md';
|
||||
const expected = 'introduction';
|
||||
expect(extractTitleFromPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle empty string', () => {
|
||||
const input = '';
|
||||
const expected = '';
|
||||
expect(extractTitleFromPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle paths with no filename', () => {
|
||||
const input = 'docs/svelte/';
|
||||
const expected = '';
|
||||
expect(extractTitleFromPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle files with hyphens but no numbers', () => {
|
||||
const input = 'docs/svelte/state-management.md';
|
||||
const expected = 'state-management';
|
||||
expect(extractTitleFromPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle files with numbers in the middle', () => {
|
||||
const input = 'docs/svelte/svelte5-features.md';
|
||||
const expected = 'svelte5-features';
|
||||
expect(extractTitleFromPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle tutorial paths', () => {
|
||||
const input = 'tutorial/01-introduction/01-hello-world.md';
|
||||
const expected = 'hello-world';
|
||||
expect(extractTitleFromPath(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle SvelteKit paths', () => {
|
||||
const input = 'docs/kit/01-routing.md';
|
||||
const expected = 'routing';
|
||||
expect(extractTitleFromPath(input)).toBe(expected);
|
||||
});
|
||||
});
|
||||
|
||||
describe('removeFrontmatter', () => {
|
||||
it('should remove valid frontmatter from content', () => {
|
||||
const input = `---
|
||||
title: Introduction
|
||||
description: Getting started guide
|
||||
---
|
||||
|
||||
# Introduction
|
||||
|
||||
This is the main content.`;
|
||||
const expected = `# Introduction
|
||||
|
||||
This is the main content.`;
|
||||
expect(removeFrontmatter(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle content without frontmatter', () => {
|
||||
const input = `# Introduction
|
||||
|
||||
This is content without frontmatter.`;
|
||||
const expected = `# Introduction
|
||||
|
||||
This is content without frontmatter.`;
|
||||
expect(removeFrontmatter(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle empty content', () => {
|
||||
const input = '';
|
||||
const expected = '';
|
||||
expect(removeFrontmatter(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle malformed frontmatter (no closing delimiter)', () => {
|
||||
const input = `---
|
||||
title: Introduction
|
||||
This is malformed frontmatter without closing delimiter
|
||||
|
||||
# Content here`;
|
||||
const expected = input; // Should return original content unchanged
|
||||
expect(removeFrontmatter(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle frontmatter with complex YAML', () => {
|
||||
const input = `---
|
||||
title: Complex Example
|
||||
tags:
|
||||
- svelte
|
||||
- tutorial
|
||||
metadata:
|
||||
author: John Doe
|
||||
date: 2024-01-15
|
||||
---
|
||||
|
||||
# Complex Example
|
||||
|
||||
Content with complex frontmatter.`;
|
||||
const expected = `# Complex Example
|
||||
|
||||
Content with complex frontmatter.`;
|
||||
expect(removeFrontmatter(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle content that starts with --- but is not frontmatter', () => {
|
||||
const input = `---
|
||||
This is not YAML frontmatter, just content that starts with ---`;
|
||||
const expected = input; // Should return original content unchanged
|
||||
expect(removeFrontmatter(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle frontmatter with empty lines', () => {
|
||||
const input = `---
|
||||
title: Introduction
|
||||
|
||||
description: A guide
|
||||
---
|
||||
|
||||
# Content`;
|
||||
const expected = `# Content`;
|
||||
expect(removeFrontmatter(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should trim whitespace after removing frontmatter', () => {
|
||||
const input = `---
|
||||
title: Introduction
|
||||
---
|
||||
|
||||
|
||||
# Content with leading whitespace`;
|
||||
const expected = `# Content with leading whitespace`;
|
||||
expect(removeFrontmatter(input)).toBe(expected);
|
||||
});
|
||||
|
||||
it('should handle frontmatter at the end of content', () => {
|
||||
const input = `---
|
||||
title: Only Frontmatter
|
||||
---`;
|
||||
const expected = ``;
|
||||
expect(removeFrontmatter(input)).toBe(expected);
|
||||
});
|
||||
});
|
||||
|
||||
describe('integration tests', () => {
|
||||
it('should work together for typical documentation workflow', () => {
|
||||
// Simulate a typical path from tarball to display
|
||||
const tarball_path = 'svelte.dev-main/apps/svelte.dev/content/docs/svelte/01-introduction.md';
|
||||
|
||||
// Clean tarball path
|
||||
const cleaned_from_tarball = cleanTarballPath(tarball_path);
|
||||
expect(cleaned_from_tarball).toBe('apps/svelte.dev/content/docs/svelte/01-introduction.md');
|
||||
|
||||
// This would be stored in DB and later cleaned for display
|
||||
const cleaned_for_display = cleanDocumentationPath(cleaned_from_tarball);
|
||||
expect(cleaned_for_display).toBe('docs/svelte/01-introduction.md');
|
||||
|
||||
// Extract title for metadata
|
||||
const title = extractTitleFromPath(cleaned_from_tarball);
|
||||
expect(title).toBe('introduction');
|
||||
});
|
||||
|
||||
it('should handle SvelteKit paths through full workflow', () => {
|
||||
const tarball_path = 'svelte.dev-main/apps/svelte.dev/content/docs/kit/01-routing.md';
|
||||
|
||||
const cleaned_from_tarball = cleanTarballPath(tarball_path);
|
||||
expect(cleaned_from_tarball).toBe('apps/svelte.dev/content/docs/kit/01-routing.md');
|
||||
|
||||
const cleaned_for_display = cleanDocumentationPath(cleaned_from_tarball);
|
||||
expect(cleaned_for_display).toBe('docs/kit/01-routing.md');
|
||||
|
||||
const title = extractTitleFromPath(cleaned_from_tarball);
|
||||
expect(title).toBe('routing');
|
||||
});
|
||||
|
||||
it('should handle tutorial paths through full workflow', () => {
|
||||
const tarball_path =
|
||||
'svelte.dev-main/apps/svelte.dev/content/tutorial/01-introduction/01-hello-world.md';
|
||||
|
||||
const cleaned_from_tarball = cleanTarballPath(tarball_path);
|
||||
expect(cleaned_from_tarball).toBe(
|
||||
'apps/svelte.dev/content/tutorial/01-introduction/01-hello-world.md',
|
||||
);
|
||||
|
||||
const cleaned_for_display = cleanDocumentationPath(cleaned_from_tarball);
|
||||
expect(cleaned_for_display).toBe('tutorial/01-introduction/01-hello-world.md');
|
||||
|
||||
const title = extractTitleFromPath(cleaned_from_tarball);
|
||||
expect(title).toBe('hello-world');
|
||||
});
|
||||
|
||||
it('should handle content processing with frontmatter removal', () => {
|
||||
const content = `---
|
||||
title: Introduction
|
||||
---
|
||||
|
||||
# Introduction
|
||||
|
||||
This is the content.`;
|
||||
|
||||
const content_without_frontmatter = removeFrontmatter(content);
|
||||
expect(content_without_frontmatter).toBe(`# Introduction
|
||||
|
||||
This is the content.`);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -1,110 +0,0 @@
|
||||
/**
|
||||
* Unified path utilities for handling documentation paths
|
||||
*/
|
||||
|
||||
/**
|
||||
* Clean a path by removing the "apps/svelte.dev/content/" prefix
|
||||
* This is used to convert database paths to display paths
|
||||
*
|
||||
* @param path - The path to clean
|
||||
* @returns The cleaned path
|
||||
*/
|
||||
export function cleanDocumentationPath(path: string): string {
|
||||
const prefix = 'apps/svelte.dev/content/';
|
||||
if (path.startsWith(prefix)) {
|
||||
return path.substring(prefix.length);
|
||||
}
|
||||
return path;
|
||||
}
|
||||
|
||||
/**
|
||||
* Clean a tarball path by removing the repository directory prefix (first segment)
|
||||
* This is used when processing files from GitHub tarballs
|
||||
*
|
||||
* @param path - The path to clean
|
||||
* @returns The cleaned path without the repo directory prefix
|
||||
*/
|
||||
export function cleanTarballPath(path: string): string {
|
||||
// Remove only the repo directory prefix (first segment)
|
||||
return path.split('/').slice(1).join('/');
|
||||
}
|
||||
|
||||
/**
|
||||
* Extract the title from a file path by removing prefixes and file extensions
|
||||
*
|
||||
* @param filePath - The file path to extract title from
|
||||
* @returns The extracted title
|
||||
*/
|
||||
export function extractTitleFromPath(filePath: string): string {
|
||||
if (!filePath) {
|
||||
return '';
|
||||
}
|
||||
|
||||
const pathParts = filePath.split('/');
|
||||
const filename = pathParts[pathParts.length - 1];
|
||||
|
||||
// Handle empty filename (e.g., paths ending with '/')
|
||||
if (!filename) {
|
||||
return '';
|
||||
}
|
||||
|
||||
// Remove .md extension and numbered prefixes
|
||||
return filename.replace('.md', '').replace(/^\d+-/, '');
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove frontmatter from markdown content using a tokenizer approach
|
||||
* Frontmatter is YAML metadata at the beginning of files between --- delimiters
|
||||
*
|
||||
* @param content - The markdown content that may contain frontmatter
|
||||
* @returns The content with frontmatter removed
|
||||
*/
|
||||
export function removeFrontmatter(content: string): string {
|
||||
if (!content || content.length === 0) {
|
||||
return content;
|
||||
}
|
||||
|
||||
// Check if content starts with frontmatter delimiter
|
||||
if (!content.startsWith('---\n')) {
|
||||
return content;
|
||||
}
|
||||
|
||||
let position = 4; // Start after the opening "---\n"
|
||||
let insideFrontmatter = true;
|
||||
let frontmatterEndOffset: number | null = null;
|
||||
|
||||
// Traverse the string character by character
|
||||
while (position < content.length && insideFrontmatter) {
|
||||
const char = content[position];
|
||||
|
||||
// Look for potential end of frontmatter: \n---
|
||||
if (char === '\n' && position + 3 < content.length) {
|
||||
const nextThree = content.substring(position + 1, position + 4);
|
||||
if (nextThree === '---') {
|
||||
// Check what comes after the closing ---
|
||||
const afterClosing = position + 4;
|
||||
|
||||
if (afterClosing >= content.length) {
|
||||
// End of string - this is valid frontmatter
|
||||
frontmatterEndOffset = content.length;
|
||||
insideFrontmatter = false;
|
||||
} else if (content[afterClosing] === '\n') {
|
||||
// Followed by newline - this is valid frontmatter
|
||||
frontmatterEndOffset = afterClosing + 1;
|
||||
insideFrontmatter = false;
|
||||
}
|
||||
// If followed by something else, it's not the end delimiter, continue searching
|
||||
}
|
||||
}
|
||||
|
||||
position++;
|
||||
}
|
||||
|
||||
// If we never found the end of frontmatter, it's malformed
|
||||
if (frontmatterEndOffset === null) {
|
||||
return content;
|
||||
}
|
||||
|
||||
// Return content after the frontmatter, trimmed
|
||||
return content.substring(frontmatterEndOffset).trim();
|
||||
}
|
||||
@@ -1,165 +0,0 @@
|
||||
export const SVELTE_5_PROMPT =
|
||||
'Always use Svelte 5 runes and Svelte 5 syntax. Runes do not need to be imported, they are globals. $state() runes are always declared using `let`, never with `const`. When passing a function to $derived, you must always use $derived.by(() => ...). Error boundaries can only catch errors during component rendering and at the top level of an $effect inside the error boundary. Error boundaries do not catch errors in onclick or other event handlers.';
|
||||
|
||||
export const DISTILLATION_PROMPT = `
|
||||
You are an expert in web development, specifically Svelte 5 and SvelteKit. Your task is to condense and distill the Svelte documentation into a concise format while preserving the most important information.
|
||||
Shorten the text information AS MUCH AS POSSIBLE while covering key concepts.
|
||||
|
||||
Focus on:
|
||||
1. Code examples with short explanations of how they work
|
||||
2. Key concepts and APIs with their usage patterns
|
||||
3. Important gotchas and best practices
|
||||
4. Patterns that developers commonly use
|
||||
|
||||
Remove:
|
||||
1. Redundant explanations
|
||||
2. Verbose content that can be simplified
|
||||
3. Marketing language
|
||||
4. Legacy or deprecated content
|
||||
5. Anything else that is not strictly necessary
|
||||
|
||||
Keep your output in markdown format. Preserve code blocks with their language annotations.
|
||||
Maintain headings but feel free to combine or restructure sections to improve clarity.
|
||||
|
||||
Make sure all code examples use Svelte 5 runes syntax ($state, $derived, $effect, etc.)
|
||||
|
||||
Keep the following Svelte 5 syntax rules in mind:
|
||||
* There is no colon (:) in event modifiers. You MUST use "onclick" instead of "on:click".
|
||||
* Runes do not need to be imported, they are globals.
|
||||
* $state() runes are always declared using let, never with const.
|
||||
* When passing a function to $derived, you must always use $derived.by(() => ...).
|
||||
* Error boundaries can only catch errors during component rendering and at the top level of an $effect inside the error boundary.
|
||||
* Error boundaries do not catch errors in onclick or other event handlers.
|
||||
|
||||
IMPORTANT: All code examples MUST come from the documentation verbatim, do NOT create new code examples. Do NOT modify existing code examples.
|
||||
IMPORTANT: Because of changes in Svelte 5 syntax, do not include content from your existing knowledge, you may only use knowledge from the documentation to condense.
|
||||
|
||||
Here is the documentation you must condense:
|
||||
|
||||
`;
|
||||
|
||||
export const SVELTE_DEVELOPER_PROMPT = `You are an expert in web development, specifically Svelte 5 and SvelteKit, with expert-level knowledge of Svelte 5, SvelteKit, and TypeScript.
|
||||
|
||||
## Core Expertise:
|
||||
|
||||
### Svelte 5 Runes & Reactivity
|
||||
- **$state**: Reactive state declaration (always use let, never const)
|
||||
- **$derived**: Computed values (always use $derived.by(() => ...) for functions)
|
||||
- **$effect**: Side effects and cleanup (runs after DOM updates)
|
||||
- **$props**: Component props with destructuring and defaults
|
||||
- **$bindable**: Two-way binding for props
|
||||
|
||||
### Critical Syntax Rules:
|
||||
${SVELTE_5_PROMPT}
|
||||
|
||||
### Additional Rules:
|
||||
- Props: let { count = 0, name } = $props()
|
||||
- Bindable: let { value = $bindable() } = $props()
|
||||
- Children: let { children } = $props()
|
||||
- Cleanup: $effect(() => { return () => cleanup() })
|
||||
- Context: setContext/getContext work with runes
|
||||
- Snippets: {#snippet name(params)} for reusable templates
|
||||
|
||||
### SvelteKit Essentials:
|
||||
- File-based routing with route groups and parameters
|
||||
- Load functions: +page.ts (universal) vs +page.server.ts (server-only)
|
||||
- Form actions in +page.server.ts with progressive enhancement
|
||||
- Layout nesting and data inheritance
|
||||
- Error and loading states with +error.svelte and loading UI
|
||||
|
||||
### TypeScript Integration:
|
||||
- Always use TypeScript for type safety
|
||||
- Properly type PageData, PageLoad, Actions, RequestHandler
|
||||
- Generic components with proper type inference
|
||||
- .svelte.ts for shared reactive state
|
||||
|
||||
## MCP Tool Usage Guide:
|
||||
|
||||
### Template Prompts (Efficient Documentation Injection):
|
||||
Use these for instant access to curated documentation sets:
|
||||
- **svelte-core**: Core Svelte 5 (introduction, runes, template syntax, styling)
|
||||
- **svelte-advanced**: Advanced Svelte 5 (special elements, runtime, misc)
|
||||
- **svelte-complete**: Complete Svelte 5 documentation
|
||||
- **sveltekit-core**: Core SvelteKit (getting started, core concepts)
|
||||
- **sveltekit-production**: Production SvelteKit (build/deploy, advanced, best practices)
|
||||
- **sveltekit-complete**: Complete SvelteKit documentation
|
||||
|
||||
### Resources Access:
|
||||
- **📦 Preset Resources**: Use svelte-llm://svelte-core, svelte-llm://svelte-advanced, svelte-llm://svelte-complete, svelte-llm://sveltekit-core, svelte-llm://sveltekit-production, svelte-llm://sveltekit-complete for curated documentation sets
|
||||
- **📄 Individual Docs**: Use svelte-llm://doc/[path] for specific documentation files
|
||||
- Access via list_resources or direct URI for browsing and reference
|
||||
|
||||
### When to use list_sections + get_documentation:
|
||||
- **Specific Topics**: When you need particular sections not covered by presets
|
||||
- **Custom Combinations**: When presets don't match the exact scope needed
|
||||
- **Deep Dives**: When you need detailed information on specific APIs
|
||||
- **Troubleshooting**: When investigating specific issues or edge cases
|
||||
|
||||
### Strategic Approach:
|
||||
1. **Start with Template Prompts**: Use template prompts (svelte-core, sveltekit-core, etc.) for immediate context injection
|
||||
2. **Browse via Resources**: Use preset resources for reading/reference during development
|
||||
3. **Supplement with Specific Docs**: Use list_sections + get_documentation only when presets don't cover your needs
|
||||
4. **Combine Efficiently**: Use multiple template prompts if you need both Svelte and SvelteKit context
|
||||
|
||||
### Documentation Fetching Priority:
|
||||
1. **Template Prompts First**: Always try relevant template prompts before individual sections
|
||||
2. **Preset Resources**: Use for browsing and reference
|
||||
3. **Individual Sections**: Only when specific content not in presets is needed
|
||||
4. **Multiple Sources**: Combine template prompts with specific sections as needed
|
||||
|
||||
## Best Practices:
|
||||
- Write production-ready TypeScript code
|
||||
- Include proper error handling and loading states
|
||||
- Consider accessibility (ARIA, keyboard navigation)
|
||||
- Optimize for performance (lazy loading, minimal reactivity)
|
||||
- Use semantic HTML and proper component composition
|
||||
- Implement proper cleanup in effects
|
||||
- Handle edge cases and provide fallbacks`;
|
||||
|
||||
// eslint-disable-next-line @typescript-eslint/naming-convention, func-style
|
||||
export const createSvelteDeveloperPromptWithTask = (task?: string): string => {
|
||||
// eslint-disable-next-line @typescript-eslint/naming-convention
|
||||
const basePrompt = SVELTE_DEVELOPER_PROMPT;
|
||||
|
||||
if (!task) {
|
||||
return (
|
||||
basePrompt +
|
||||
`
|
||||
|
||||
## Your Approach:
|
||||
When helping with Svelte/SvelteKit development:
|
||||
1. **Use Template Prompts**: Start with relevant template prompts (svelte-core, sveltekit-core, etc.) for immediate context
|
||||
2. **Supplement as Needed**: Use list_sections + get_documentation only for content not covered by templates
|
||||
3. **Provide Complete Solutions**: Include working TypeScript code with proper types
|
||||
4. **Explain Trade-offs**: Discuss architectural decisions and alternatives
|
||||
5. **Optimize**: Suggest performance improvements and best practices`
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
basePrompt +
|
||||
`
|
||||
|
||||
## Current Task:
|
||||
${task}
|
||||
|
||||
## Task-Specific Approach:
|
||||
1. **Inject Relevant Context**: Use appropriate template prompts based on "${task.substring(0, 50)}...":
|
||||
- Component tasks: Use svelte-core for runes, template syntax
|
||||
- Advanced features: Use svelte-advanced for special elements, runtime
|
||||
- Full applications: Use svelte-complete + sveltekit-core/complete
|
||||
- Production apps: Use sveltekit-production for deployment, best practices
|
||||
2. **Supplement with Specific Docs**: Use list_sections + get_documentation only if templates don't cover specific needs
|
||||
3. **Design Architecture**:
|
||||
- Component structure and composition
|
||||
- State management approach
|
||||
- TypeScript types and interfaces
|
||||
- Error handling strategy
|
||||
4. **Implement Solution**:
|
||||
- Complete, working code
|
||||
- Proper types and error boundaries
|
||||
- Performance optimizations
|
||||
- Accessibility considerations
|
||||
5. **Explain Implementation**: Provide rationale for choices and discuss alternatives`
|
||||
);
|
||||
};
|
||||
@@ -1,8 +0,0 @@
|
||||
import { json } from '@sveltejs/kit';
|
||||
import type { RequestHandler } from './$types';
|
||||
import { fetchRepositoryTarball } from '$lib/fetchMarkdown';
|
||||
|
||||
export const GET: RequestHandler = async () => {
|
||||
const tarball_buffer = await fetchRepositoryTarball('sveltejs', 'svelte.dev');
|
||||
return json({ data: tarball_buffer });
|
||||
};
|
||||
17
package.json
17
package.json
@@ -4,7 +4,6 @@
|
||||
"description": "The official Svelte MCP server implementation",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "pnpm --filter mcp-remote dev",
|
||||
"build": "pnpm -r run build",
|
||||
"check": "pnpm -r run check",
|
||||
"format": "prettier --write .",
|
||||
@@ -26,27 +25,13 @@
|
||||
"@eslint/compat": "^1.3.2",
|
||||
"@eslint/js": "^9.36.0",
|
||||
"@modelcontextprotocol/inspector": "^0.16.7",
|
||||
"@sveltejs/adapter-vercel": "^5.6.3",
|
||||
"@sveltejs/kit": "^2.22.0",
|
||||
"@sveltejs/vite-plugin-svelte": "^6.0.0",
|
||||
"@types/eslint-scope": "^8.3.2",
|
||||
"@types/estree": "^1.0.8",
|
||||
"@types/node": "^24.3.1",
|
||||
"@types/tar-stream": "^3.1.4",
|
||||
"@typescript-eslint/types": "^8.43.0",
|
||||
"dotenv": "^17.2.2",
|
||||
"drizzle-kit": "^0.30.2",
|
||||
"drizzle-orm": "^0.40.0",
|
||||
"eslint": "^9.36.0",
|
||||
"eslint-config-prettier": "^10.0.1",
|
||||
"eslint-plugin-import": "^2.32.0",
|
||||
"eslint-plugin-svelte": "^3.12.3",
|
||||
"globals": "^16.0.0",
|
||||
"minimatch": "^10.0.3",
|
||||
"prettier": "^3.4.2",
|
||||
"prettier-plugin-svelte": "^3.3.3",
|
||||
"svelte": "^5.0.0",
|
||||
"svelte-check": "^4.0.0",
|
||||
"tar-stream": "^3.1.7",
|
||||
"typescript": "^5.0.0",
|
||||
"typescript-eslint": "^8.44.1",
|
||||
"vitest": "^3.2.3"
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { blob, integer, sqliteTable, text } from 'drizzle-orm/sqlite-core';
|
||||
import { integer, sqliteTable, text } from 'drizzle-orm/sqlite-core';
|
||||
import { float_32_array } from './utils.js';
|
||||
|
||||
/**
|
||||
@@ -51,7 +51,7 @@ export const distillation_jobs = sqliteTable('distillation_jobs', {
|
||||
|
||||
export const content = sqliteTable('content', {
|
||||
id: integer('id').primaryKey(),
|
||||
path: text('path').notNull().unique(),
|
||||
path: text('path').notNull(),
|
||||
filename: text('filename').notNull(),
|
||||
content: text('content').notNull(),
|
||||
size_bytes: integer('size_bytes').notNull(),
|
||||
@@ -67,7 +67,7 @@ export const content = sqliteTable('content', {
|
||||
|
||||
export const content_distilled = sqliteTable('content_distilled', {
|
||||
id: integer('id').primaryKey(),
|
||||
path: text('path').notNull().unique(),
|
||||
path: text('path').notNull(),
|
||||
filename: text('filename').notNull(),
|
||||
content: text('content').notNull(),
|
||||
size_bytes: integer('size_bytes').notNull(),
|
||||
@@ -80,17 +80,3 @@ export const content_distilled = sqliteTable('content_distilled', {
|
||||
.notNull()
|
||||
.$defaultFn(() => new Date()),
|
||||
});
|
||||
|
||||
export const cache = sqliteTable('cache', {
|
||||
id: integer('id').primaryKey(),
|
||||
cache_key: text('cache_key').notNull().unique(),
|
||||
data: blob('data', { mode: 'buffer' }).notNull(),
|
||||
size_bytes: integer('size_bytes').notNull(),
|
||||
expires_at: integer('expires_at', { mode: 'timestamp' }).notNull(),
|
||||
created_at: integer('created_at', { mode: 'timestamp' })
|
||||
.notNull()
|
||||
.$defaultFn(() => new Date()),
|
||||
updated_at: integer('updated_at', { mode: 'timestamp' })
|
||||
.notNull()
|
||||
.$defaultFn(() => new Date()),
|
||||
});
|
||||
|
||||
@@ -109,7 +109,7 @@ describe('add_autofixers_issues', () => {
|
||||
});
|
||||
|
||||
describe.each([{ method: 'set' }, { method: 'update' }])(
|
||||
'wrong_property_access_state ($method)',
|
||||
'set_or_update_state ($method)',
|
||||
({ method }) => {
|
||||
it(`should add suggestions when using .${method}() on a stateful variable with a literal init`, () => {
|
||||
const content = run_autofixers_on_code(`
|
||||
@@ -203,87 +203,6 @@ describe('add_autofixers_issues', () => {
|
||||
},
|
||||
);
|
||||
|
||||
describe.each([{ property: '$' }])(
|
||||
'wrong_property_access_state property ($property)',
|
||||
async ({ property }) => {
|
||||
it(`should add suggestions when reading .${property} on a stateful variable with a literal init`, () => {
|
||||
const content = run_autofixers_on_code(`
|
||||
<script>
|
||||
const count = $state(0);
|
||||
function read_count() {
|
||||
count.${property};
|
||||
}
|
||||
</script>`);
|
||||
|
||||
expect(content.suggestions.length).toBeGreaterThanOrEqual(1);
|
||||
expect(content.suggestions).toContain(
|
||||
`You are trying to read the stateful variable "count" using "${property}". stateful variables should be read just by accessing them like normal variable, do not use properties to read them.`,
|
||||
);
|
||||
});
|
||||
|
||||
it(`should add suggestions when reading .${property} on a stateful variable with an array init`, () => {
|
||||
const content = run_autofixers_on_code(`
|
||||
<script>
|
||||
const count = $state([1]);
|
||||
function read_count() {
|
||||
count.${property};
|
||||
}
|
||||
</script>`);
|
||||
|
||||
expect(content.suggestions.length).toBeGreaterThanOrEqual(1);
|
||||
expect(content.suggestions).toContain(
|
||||
`You are trying to read the stateful variable "count" using "${property}". stateful variables should be read just by accessing them like normal variable, do not use properties to read them.`,
|
||||
);
|
||||
});
|
||||
|
||||
it(`should add suggestions when reading .${property} on a stateful variable with conditional if it's not sure if the property could actually be present on the variable ($state({}))`, () => {
|
||||
const content = run_autofixers_on_code(`
|
||||
<script>
|
||||
const count = $state({ value: 0 });
|
||||
function read_count() {
|
||||
count.${property};
|
||||
}
|
||||
</script>`);
|
||||
|
||||
expect(content.suggestions.length).toBeGreaterThanOrEqual(1);
|
||||
expect(content.suggestions).toContain(
|
||||
`You are trying to read the stateful variable "count" using "${property}". stateful variables should be read just by accessing them like normal variable, do not use properties to read them. However I can't verify if "count" is a state variable of an object or a class with a "${property}" property on it. Please verify that before updating the code to use a normal access`,
|
||||
);
|
||||
});
|
||||
|
||||
it(`should add suggestions when reading .${property} on a stateful variable with conditional if it's not sure if the property could actually be present on the variable ($state(new Class()))`, () => {
|
||||
const content = run_autofixers_on_code(`
|
||||
<script>
|
||||
const count = $state(new Class());
|
||||
function read_count() {
|
||||
count.${property};
|
||||
}
|
||||
</script>`);
|
||||
|
||||
expect(content.suggestions.length).toBeGreaterThanOrEqual(1);
|
||||
expect(content.suggestions).toContain(
|
||||
`You are trying to read the stateful variable "count" using "${property}". stateful variables should be read just by accessing them like normal variable, do not use properties to read them. However I can't verify if "count" is a state variable of an object or a class with a "${property}" property on it. Please verify that before updating the code to use a normal access`,
|
||||
);
|
||||
});
|
||||
|
||||
it(`should add suggestions when reading .${property} on a stateful variable with conditional if it's not sure if the property could actually be present on the variable ($state(variable_name))`, () => {
|
||||
const content = run_autofixers_on_code(`
|
||||
<script>
|
||||
const { init } = $props();
|
||||
const count = $state(init);
|
||||
function read_count() {
|
||||
count.${property};
|
||||
}
|
||||
</script>`);
|
||||
|
||||
expect(content.suggestions.length).toBeGreaterThanOrEqual(1);
|
||||
expect(content.suggestions).toContain(
|
||||
`You are trying to read the stateful variable "count" using "${property}". stateful variables should be read just by accessing them like normal variable, do not use properties to read them. However I can't verify if "count" is a state variable of an object or a class with a "${property}" property on it. Please verify that before updating the code to use a normal access`,
|
||||
);
|
||||
});
|
||||
},
|
||||
);
|
||||
|
||||
describe('imported_runes', () => {
|
||||
describe.each([{ source: 'svelte' }, { source: 'svelte/runes' }])(
|
||||
'from "$source"',
|
||||
@@ -455,4 +374,177 @@ describe('add_autofixers_issues', () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('suggest_attachments', () => {
|
||||
describe('bind:this', () => {
|
||||
it('should add suggestions when using bind:this on an element', () => {
|
||||
const content = run_autofixers_on_code(`
|
||||
<script>
|
||||
let a = $state();
|
||||
</script>
|
||||
|
||||
<a bind:this={a} />`);
|
||||
|
||||
expect(content.suggestions.length).toBeGreaterThanOrEqual(1);
|
||||
expect(content.suggestions).toContain(
|
||||
'The usage of `bind:this` can often be replaced with an easier to read `action` or even better an `attachment`. Consider using the latter if possible.',
|
||||
);
|
||||
});
|
||||
|
||||
it('should not add suggestions when using bind:this on a component', () => {
|
||||
const content = run_autofixers_on_code(`
|
||||
<script>
|
||||
import Child from './Child.svelte';
|
||||
let a = $state();
|
||||
</script>
|
||||
|
||||
<Child bind:this={a} />`);
|
||||
|
||||
expect(content.suggestions).not.toContain(
|
||||
'The usage of `bind:this` can often be replaced with an easier to read `action` or even better an `attachment`. Consider using the latter if possible.',
|
||||
);
|
||||
});
|
||||
|
||||
it('should not add suggestions when using bind:this on a component nested in an element', () => {
|
||||
const content = run_autofixers_on_code(`
|
||||
<script>
|
||||
import Child from './Child.svelte';
|
||||
let a = $state();
|
||||
</script>
|
||||
|
||||
<div>
|
||||
<Child bind:this={a} />
|
||||
</div>`);
|
||||
|
||||
expect(content.suggestions).not.toContain(
|
||||
'The usage of `bind:this` can often be replaced with an easier to read `action` or even better an `attachment`. Consider using the latter if possible.',
|
||||
);
|
||||
});
|
||||
|
||||
it('should add suggestions but not suggest attachments when using bind:this on an element and the desired svelte version is 4', () => {
|
||||
const content = run_autofixers_on_code(
|
||||
`
|
||||
<script>
|
||||
let a;
|
||||
</script>
|
||||
|
||||
<a bind:this={a} />`,
|
||||
4,
|
||||
);
|
||||
|
||||
expect(content.suggestions.length).toBeGreaterThanOrEqual(1);
|
||||
expect(content.suggestions).toContain(
|
||||
'The usage of `bind:this` can often be replaced with an easier to read `action`. Consider using the latter if possible.',
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('use:', () => {
|
||||
it('should add suggestions when using use: on an element and the action is declared as a function', () => {
|
||||
const content = run_autofixers_on_code(
|
||||
`<script>
|
||||
function my_action(node) {
|
||||
// do something with the node
|
||||
}
|
||||
</script>
|
||||
|
||||
<a use:my_action />`,
|
||||
);
|
||||
|
||||
expect(content.suggestions.length).toBeGreaterThanOrEqual(1);
|
||||
expect(content.suggestions).toContain(
|
||||
'Consider using an `attachment` instead of an `action` for "my_action".',
|
||||
);
|
||||
});
|
||||
|
||||
it('should add suggestions when using use: on an element and the action is declared as a variable', () => {
|
||||
const content = run_autofixers_on_code(
|
||||
`<script>
|
||||
const my_action = (node) => {
|
||||
// do something with the node
|
||||
}
|
||||
</script>
|
||||
|
||||
<a use:my_action />`,
|
||||
);
|
||||
|
||||
expect(content.suggestions.length).toBeGreaterThanOrEqual(1);
|
||||
expect(content.suggestions).toContain(
|
||||
'Consider using an `attachment` instead of an `action` for "my_action".',
|
||||
);
|
||||
});
|
||||
|
||||
it('should add suggestions when using use: on an element and the action is declared as an object', () => {
|
||||
const content = run_autofixers_on_code(
|
||||
`<script>
|
||||
const my_action = {
|
||||
action: (node) => {
|
||||
// do something with the node
|
||||
}
|
||||
};
|
||||
</script>
|
||||
|
||||
<a use:my_action.action />`,
|
||||
);
|
||||
|
||||
expect(content.suggestions.length).toBeGreaterThanOrEqual(1);
|
||||
expect(content.suggestions).toContain(
|
||||
'Consider using an `attachment` instead of an `action` for "my_action".',
|
||||
);
|
||||
});
|
||||
|
||||
it('should not add suggestions when using use: on an element and the desired svelte version is 4', () => {
|
||||
const content = run_autofixers_on_code(
|
||||
`<script>
|
||||
function my_action(node) {
|
||||
// do something with the node
|
||||
}
|
||||
</script>
|
||||
|
||||
<a use:my_action />`,
|
||||
4,
|
||||
);
|
||||
|
||||
expect(content.suggestions).not.toContain(
|
||||
'Consider using an `attachment` instead of an `action` for "my_action".',
|
||||
);
|
||||
});
|
||||
|
||||
it('should not add suggestions when using use: on an element and the action comes from an import', () => {
|
||||
const content = run_autofixers_on_code(
|
||||
`<script>
|
||||
import { my_action } from './actions.js';
|
||||
</script>
|
||||
|
||||
<a use:my_action />`,
|
||||
);
|
||||
|
||||
expect(content.suggestions).not.toContain(
|
||||
'Consider using an `attachment` instead of an `action` for "my_action".',
|
||||
);
|
||||
});
|
||||
|
||||
it('should not add suggestions when using use: on an element and the action comes from the props', () => {
|
||||
const content = run_autofixers_on_code(
|
||||
`<script>
|
||||
const { my_action } = $props();
|
||||
</script>
|
||||
|
||||
<a use:my_action />`,
|
||||
);
|
||||
|
||||
expect(content.suggestions).not.toContain(
|
||||
'Consider using an `attachment` instead of an `action` for "my_action".',
|
||||
);
|
||||
});
|
||||
|
||||
it('should not add suggestions when using use: on an element and the action comes from a global variable', () => {
|
||||
const content = run_autofixers_on_code(`<a use:my_action />`);
|
||||
|
||||
expect(content.suggestions).not.toContain(
|
||||
'Consider using an `attachment` instead of an `action` for "my_action".',
|
||||
);
|
||||
});
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,14 +1,10 @@
|
||||
import type { AssignmentExpression, Identifier, Node, UpdateExpression } from 'estree';
|
||||
import type { Autofixer, AutofixerState } from './index.js';
|
||||
import type { Autofixer, AutofixerState } from '.';
|
||||
import { left_most_id } from '../ast/utils.js';
|
||||
import type { AST } from 'svelte-eslint-parser';
|
||||
import type { SvelteNode } from 'svelte-eslint-parser/lib/ast';
|
||||
import type { Context } from 'zimmerframe';
|
||||
|
||||
function run_if_in_effect(
|
||||
path: (Node | AST.SvelteNode)[],
|
||||
state: AutofixerState,
|
||||
to_run: () => void,
|
||||
) {
|
||||
function run_if_in_effect(path: (Node | SvelteNode)[], state: AutofixerState, to_run: () => void) {
|
||||
const in_effect = path.findLast(
|
||||
(node) =>
|
||||
node.type === 'CallExpression' &&
|
||||
@@ -29,7 +25,7 @@ function run_if_in_effect(
|
||||
|
||||
function visitor(
|
||||
node: UpdateExpression | AssignmentExpression,
|
||||
{ state, path }: Context<Node | AST.SvelteNode, AutofixerState>,
|
||||
{ state, path }: Context<Node | SvelteNode, AutofixerState>,
|
||||
) {
|
||||
run_if_in_effect(path, state, () => {
|
||||
function check_if_stateful_id(id: Identifier) {
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import type { Identifier, PrivateIdentifier } from 'estree';
|
||||
import type { Autofixer } from './index.js';
|
||||
import type { Autofixer } from '.';
|
||||
|
||||
export const derived_with_function: Autofixer = {
|
||||
CallExpression(node, { state, path }) {
|
||||
@@ -7,15 +7,15 @@ export const derived_with_function: Autofixer = {
|
||||
node.callee.type === 'Identifier' &&
|
||||
node.callee.name === '$derived' &&
|
||||
state.parsed.is_rune(node, ['$derived']) &&
|
||||
(node.arguments[0]?.type === 'ArrowFunctionExpression' ||
|
||||
node.arguments[0]?.type === 'FunctionExpression')
|
||||
(node.arguments[0].type === 'ArrowFunctionExpression' ||
|
||||
node.arguments[0].type === 'FunctionExpression')
|
||||
) {
|
||||
const parent = path[path.length - 1];
|
||||
let variable_id: Identifier | PrivateIdentifier | undefined;
|
||||
if (parent?.type === 'VariableDeclarator' && parent.id.type === 'Identifier') {
|
||||
if (parent.type === 'VariableDeclarator' && parent.id.type === 'Identifier') {
|
||||
// const something = $derived(...)
|
||||
variable_id = parent.id;
|
||||
} else if (parent?.type === 'PropertyDefinition') {
|
||||
} else if (parent.type === 'PropertyDefinition') {
|
||||
// class X { something = $derived(...) }
|
||||
variable_id =
|
||||
parent.key.type === 'Identifier'
|
||||
@@ -23,7 +23,7 @@ export const derived_with_function: Autofixer = {
|
||||
: parent.key.type === 'PrivateIdentifier'
|
||||
? parent.key
|
||||
: undefined;
|
||||
} else if (parent?.type === 'AssignmentExpression') {
|
||||
} else if (parent.type === 'AssignmentExpression') {
|
||||
// this.something = $derived(...)
|
||||
variable_id =
|
||||
parent.left.type === 'MemberExpression'
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { base_runes } from '../../../constants.js';
|
||||
import type { Autofixer } from './index.js';
|
||||
import type { Autofixer } from '.';
|
||||
|
||||
const dollarless_runes = base_runes.map((r) => r.replace('$', ''));
|
||||
|
||||
|
||||
@@ -12,7 +12,8 @@ export type AutofixerState = {
|
||||
export type Autofixer = Visitors<Node | AST.SvelteNode, AutofixerState>;
|
||||
|
||||
export * from './assign-in-effect.js';
|
||||
export * from './wrong-property-access-state.js';
|
||||
export * from './set-or-update-state.js';
|
||||
export * from './imported-runes.js';
|
||||
export * from './derived-with-function.js';
|
||||
export * from './use-runes-instead-of-store.js';
|
||||
export * from './suggest-attachments.js';
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
import type { Autofixer } from '.';
|
||||
import { left_most_id } from '../ast/utils.js';
|
||||
|
||||
const UPDATE_PROPERTIES = ['set', 'update'];
|
||||
|
||||
export const set_or_update_state: Autofixer = {
|
||||
MemberExpression(node, { state, next, path }) {
|
||||
const parent = path[path.length - 1];
|
||||
if (
|
||||
parent.type === 'CallExpression' &&
|
||||
parent.callee === node &&
|
||||
node.property.type === 'Identifier' &&
|
||||
UPDATE_PROPERTIES.includes(node.property.name)
|
||||
) {
|
||||
const id = left_most_id(node);
|
||||
if (id) {
|
||||
const reference = state.parsed.find_reference_by_id(id);
|
||||
const definition = reference?.resolved?.defs[0];
|
||||
if (definition && definition.type === 'Variable') {
|
||||
const init = definition.node.init;
|
||||
if (
|
||||
init?.type === 'CallExpression' &&
|
||||
state.parsed.is_rune(init, ['$state', '$state.raw'])
|
||||
) {
|
||||
let suggestion = `You are trying to update the stateful variable "${id.name}" using "${node.property.name}". stateful variables should be updated with a normal assignment/mutation, do not use methods to update them.`;
|
||||
const argument = init.arguments[0];
|
||||
if (!argument || (argument.type !== 'Literal' && argument.type !== 'ArrayExpression')) {
|
||||
suggestion += ` However I can't verify if "${id.name}" is a state variable of an object or a class with a "${node.property.name}" method on it. Please verify that before updating the code to use a normal assignment`;
|
||||
}
|
||||
state.output.suggestions.push(suggestion);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
next();
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,47 @@
|
||||
import type { Identifier } from 'estree';
|
||||
import type { Autofixer } from './index.js';
|
||||
import { left_most_id } from '../ast/utils.js';
|
||||
|
||||
export const suggest_attachments: Autofixer = {
|
||||
SvelteDirective(node, { state, next, path }) {
|
||||
if (node.kind === 'Binding' && node.key.name.name === 'this') {
|
||||
const parent_element = path.findLast((p) => p.type === 'SvelteElement');
|
||||
if (parent_element?.kind === 'html' && parent_element.startTag.attributes.includes(node)) {
|
||||
let better_an_attachment = ` or even better an \`attachment\``;
|
||||
if (state.desired_svelte_version === 4) {
|
||||
better_an_attachment = ``;
|
||||
}
|
||||
state.output.suggestions.push(
|
||||
`The usage of \`bind:this\` can often be replaced with an easier to read \`action\`${better_an_attachment}. Consider using the latter if possible.`,
|
||||
);
|
||||
}
|
||||
} else if (node.kind === 'Action' && state.desired_svelte_version === 5) {
|
||||
let id: Identifier | null = null;
|
||||
if (node.key.name.type === 'Identifier') {
|
||||
id = node.key.name;
|
||||
} else if (node.key.name.type === 'MemberExpression') {
|
||||
id = left_most_id(node.key.name);
|
||||
}
|
||||
if (id) {
|
||||
const reference = state.parsed.find_reference_by_id(id);
|
||||
const definition = reference?.resolved?.defs[0];
|
||||
console.log(definition);
|
||||
if (
|
||||
definition &&
|
||||
(definition.type === 'Variable' ||
|
||||
!(definition.type === 'ImportBinding' || definition.type === 'Parameter')) &&
|
||||
!(
|
||||
definition.type === 'Variable' &&
|
||||
definition.node.init?.type === 'CallExpression' &&
|
||||
state.parsed.is_rune(definition.node.init, ['$props'])
|
||||
)
|
||||
) {
|
||||
state.output.suggestions.push(
|
||||
`Consider using an \`attachment\` instead of an \`action\` for "${id.name}".`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
next();
|
||||
},
|
||||
};
|
||||
@@ -1,4 +1,4 @@
|
||||
import type { Autofixer } from './index.js';
|
||||
import type { Autofixer } from '.';
|
||||
|
||||
export const use_runes_instead_of_store: Autofixer = {
|
||||
ImportDeclaration(node, { state, next }) {
|
||||
|
||||
@@ -1,41 +0,0 @@
|
||||
import type { Autofixer } from './index.js';
|
||||
import { left_most_id } from '../ast/utils.js';
|
||||
|
||||
const UPDATE_PROPERTIES = new Set(['set', 'update', '$']);
|
||||
const METHODS = new Set(['set', 'update']);
|
||||
|
||||
export const wrong_property_access_state: Autofixer = {
|
||||
MemberExpression(node, { state, next, path }) {
|
||||
const parent = path[path.length - 1];
|
||||
let is_property = false;
|
||||
if (
|
||||
node.property.type === 'Identifier' &&
|
||||
((is_property = !METHODS.has(node.property.name)) ||
|
||||
(parent?.type === 'CallExpression' && parent.callee === node)) &&
|
||||
UPDATE_PROPERTIES.has(node.property.name)
|
||||
) {
|
||||
const id = left_most_id(node);
|
||||
if (id) {
|
||||
const reference = state.parsed.find_reference_by_id(id);
|
||||
const definition = reference?.resolved?.defs[0];
|
||||
if (definition && definition.type === 'Variable') {
|
||||
const init = definition.node.init;
|
||||
if (
|
||||
init?.type === 'CallExpression' &&
|
||||
state.parsed.is_rune(init, ['$state', '$state.raw'])
|
||||
) {
|
||||
let suggestion = is_property
|
||||
? `You are trying to read the stateful variable "${id.name}" using "${node.property.name}". stateful variables should be read just by accessing them like normal variable, do not use properties to read them.`
|
||||
: `You are trying to update the stateful variable "${id.name}" using "${node.property.name}". stateful variables should be updated with a normal assignment/mutation, do not use methods to update them.`;
|
||||
const argument = init.arguments[0];
|
||||
if (!argument || (argument.type !== 'Literal' && argument.type !== 'ArrayExpression')) {
|
||||
suggestion += ` However I can't verify if "${id.name}" is a state variable of an object or a class with a "${node.property.name}" ${is_property ? 'property' : 'method'} on it. Please verify that before updating the code to use a normal ${is_property ? 'access' : 'assignment'}`;
|
||||
}
|
||||
state.output.suggestions.push(suggestion);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
next();
|
||||
},
|
||||
};
|
||||
@@ -13,7 +13,7 @@ async function compress_and_encode_text(input: string) {
|
||||
} else {
|
||||
for (let i = 0; i < value.length; i++) {
|
||||
// decoding as utf-8 will make btoa reject the string
|
||||
buffer += String.fromCharCode(value[i]!);
|
||||
buffer += String.fromCharCode(value[i]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -61,11 +61,6 @@ export function svelte_autofixer(server: SvelteMcp) {
|
||||
content.issues.push(
|
||||
`${error.message} at line ${error.start?.line}, column ${error.start?.column}`,
|
||||
);
|
||||
if (error.message.includes('js_parse_error')) {
|
||||
content.suggestions.push(
|
||||
"The code can't be compiled because a Javascript parse error. In case you are using runes like this `$state variable_name = 3;` or `$derived variable_name = 3 * count` that's not how runes are used. You need to use them as function calls without importing them: `const variable_name = $state(3)` and `const variable_name = $derived(3 * count)`.",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
if (content.issues.length > 0 || content.suggestions.length > 0) {
|
||||
|
||||
@@ -1,16 +0,0 @@
|
||||
# @sveltejs/mcp
|
||||
|
||||
The CLI version of the Svelte MCP.
|
||||
|
||||
You can run it directly with
|
||||
|
||||
```bash
|
||||
npx @sveltejs/mcp
|
||||
```
|
||||
|
||||
or install it and then run it
|
||||
|
||||
```bash
|
||||
pnpm i @sveltejs/mcp
|
||||
pnpm svelte-mcp
|
||||
```
|
||||
@@ -1,42 +0,0 @@
|
||||
{
|
||||
"name": "@sveltejs/mcp",
|
||||
"version": "0.0.1",
|
||||
"type": "module",
|
||||
"license": "MIT",
|
||||
"homepage": "https://github.com/sveltejs/mcp#readme",
|
||||
"bugs": {
|
||||
"url": "https://github.com/sveltejs/mcp/issues"
|
||||
},
|
||||
"bin": {
|
||||
"svelte-mcp": "./dist/index.js"
|
||||
},
|
||||
"repository": {
|
||||
"type": "git",
|
||||
"url": "git+https://github.com/sveltejs/mcp.git",
|
||||
"path": "packages/mcp-stdio"
|
||||
},
|
||||
"files": [
|
||||
"dist"
|
||||
],
|
||||
"publishConfig": {
|
||||
"access": "public"
|
||||
},
|
||||
"scripts": {
|
||||
"build": "tsdown",
|
||||
"dev": "tsdown --watch",
|
||||
"test": "vitest",
|
||||
"check": "tsc --noEmit"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@sveltejs/mcp-server": "workspace:^",
|
||||
"@tmcp/transport-stdio": "^0.3.0",
|
||||
"@types/node": "^22.15.17",
|
||||
"publint": "^0.3.13",
|
||||
"tsdown": "^0.11.9",
|
||||
"typescript": "^5.8.3",
|
||||
"vitest": "^3.1.3"
|
||||
},
|
||||
"dependencies": {
|
||||
"eslint": "^9.36.0"
|
||||
}
|
||||
}
|
||||
@@ -1,7 +0,0 @@
|
||||
#! /usr/bin/env node
|
||||
import { server } from '@sveltejs/mcp-server';
|
||||
import { StdioTransport } from '@tmcp/transport-stdio';
|
||||
|
||||
const transport = new StdioTransport(server);
|
||||
|
||||
transport.listen();
|
||||
@@ -1,5 +0,0 @@
|
||||
{
|
||||
"extends": "../../tsconfig.json",
|
||||
"include": ["src"],
|
||||
"exclude": ["node_modules", "dist"]
|
||||
}
|
||||
@@ -1,21 +0,0 @@
|
||||
import { defineConfig } from 'tsdown';
|
||||
|
||||
export default defineConfig([
|
||||
{
|
||||
entry: ['./src/index.ts'],
|
||||
platform: 'node',
|
||||
define: {
|
||||
// some eslint-plugin-svelte code expects __filename to exists but in an ESM environment it does not.
|
||||
__filename: 'import.meta.filename',
|
||||
},
|
||||
// we need eslint at runtime but the bundler doesn't bundle `require`'s which `eslint-plugin-svelte` uses to require
|
||||
// `eslint/use-at-your-own-risk`. If we didn't have `eslint` as an actual dependency and didn't externalize it
|
||||
// the require would fail once executed in a project without eslint installed.
|
||||
external: ['eslint'],
|
||||
publint: true,
|
||||
dts: false,
|
||||
treeshake: true,
|
||||
clean: true,
|
||||
target: 'esnext',
|
||||
},
|
||||
]);
|
||||
@@ -1,3 +0,0 @@
|
||||
import { defineConfig } from 'vitest/config';
|
||||
|
||||
export default defineConfig({});
|
||||
998
pnpm-lock.yaml
generated
998
pnpm-lock.yaml
generated
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user