mirror of
https://github.com/github/spec-kit.git
synced 2026-07-03 12:28:06 +08:00
feat(extensions,presets): authenticate GitHub-hosted catalog and download requests with GITHUB_TOKEN/GH_TOKEN (#2331)
* feat(extensions,presets): authenticate GitHub-hosted catalog and download requests with GITHUB_TOKEN/GH_TOKEN Squashed from #2087 (original author: @anasseth). Adds GitHub-token authentication to extension and preset catalog fetching and ZIP downloads so private GitHub repos work when GITHUB_TOKEN/GH_TOKEN is set, while preventing credential leakage to non-GitHub hosts. - Introduces shared _github_http module with build_github_request() and open_github_url() helpers - Routes ExtensionCatalog and PresetCatalog network calls through GitHub-auth-aware opener - Adds comprehensive unit/integration tests for auth header behavior - Updates user docs for both extensions and presets Co-authored-by: anasseth <16745089+anasseth@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(auth): address review feedback from #2087 - Fix redirect handler to preserve Authorization on GitHub-to-GitHub redirects (e.g. github.com → codeload.github.com). The previous implementation relied on super().redirect_request() which strips auth on cross-host redirects, breaking private repo archive downloads. - Add codeload.github.com to documented host lists in both EXTENSION-USER-GUIDE.md and presets/README.md - Add redirect auth-preservation and auth-stripping tests Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(auth): use Bearer scheme instead of token for consistency Aligns with the rest of the codebase (e.g. __init__.py:1721) and GitHub's current API guidance. Updates all test assertions accordingly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address second round of Copilot review feedback - Fix docstring to say Bearer instead of token (matches implementation) - Remove unused imports/fixtures from redirect tests (GITHUB_HOSTS, MagicMock, temp_dir, monkeypatch) - Replace __import__('io').BytesIO() with normal import io pattern in test_presets.py Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: anasseth <16745089+anasseth@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
@@ -423,7 +423,7 @@ In addition to extension-specific environment variables (`SPECKIT_{EXT_ID}_*`),
|
|||||||
| Variable | Description | Default |
|
| Variable | Description | Default |
|
||||||
|----------|-------------|---------|
|
|----------|-------------|---------|
|
||||||
| `SPECKIT_CATALOG_URL` | Override the full catalog stack with a single URL (backward compat) | Built-in default stack |
|
| `SPECKIT_CATALOG_URL` | Override the full catalog stack with a single URL (backward compat) | Built-in default stack |
|
||||||
| `GH_TOKEN` / `GITHUB_TOKEN` | GitHub API token for downloads | None |
|
| `GH_TOKEN` / `GITHUB_TOKEN` | GitHub token for authenticated requests to GitHub-hosted URLs (`raw.githubusercontent.com`, `github.com`, `api.github.com`, `codeload.github.com`). Required when your catalog JSON or extension ZIPs are hosted in a private GitHub repository. | None |
|
||||||
|
|
||||||
#### Example: Using a custom catalog for testing
|
#### Example: Using a custom catalog for testing
|
||||||
|
|
||||||
@@ -435,6 +435,21 @@ export SPECKIT_CATALOG_URL="http://localhost:8000/catalog.json"
|
|||||||
export SPECKIT_CATALOG_URL="https://example.com/staging/catalog.json"
|
export SPECKIT_CATALOG_URL="https://example.com/staging/catalog.json"
|
||||||
```
|
```
|
||||||
|
|
||||||
|
#### Example: Using a private GitHub-hosted catalog
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Authenticate with a token (gh CLI, PAT, or GITHUB_TOKEN in CI)
|
||||||
|
export GITHUB_TOKEN=$(gh auth token)
|
||||||
|
|
||||||
|
# Search a private catalog added via `specify extension catalog add`
|
||||||
|
specify extension search jira
|
||||||
|
|
||||||
|
# Install from a private catalog
|
||||||
|
specify extension add jira-sync
|
||||||
|
```
|
||||||
|
|
||||||
|
The token is attached automatically to requests targeting GitHub domains. Non-GitHub catalog URLs are always fetched without credentials.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Extension Catalogs
|
## Extension Catalogs
|
||||||
|
|||||||
@@ -123,9 +123,25 @@ See [scaffold/](scaffold/) for a scaffold you can copy to create your own preset
|
|||||||
|
|
||||||
## Environment Variables
|
## Environment Variables
|
||||||
|
|
||||||
| Variable | Description |
|
| Variable | Description | Default |
|
||||||
|----------|-------------|
|
|----------|-------------|---------|
|
||||||
| `SPECKIT_PRESET_CATALOG_URL` | Override the catalog URL (replaces all defaults) |
|
| `SPECKIT_PRESET_CATALOG_URL` | Override the full catalog stack with a single URL (replaces all defaults) | Built-in default stack |
|
||||||
|
| `GH_TOKEN` / `GITHUB_TOKEN` | GitHub token for authenticated requests to GitHub-hosted URLs (`raw.githubusercontent.com`, `github.com`, `api.github.com`, `codeload.github.com`). Required when your catalog JSON or preset ZIPs are hosted in a private GitHub repository. | None |
|
||||||
|
|
||||||
|
#### Example: Using a private GitHub-hosted catalog
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Authenticate with a token (gh CLI, PAT, or GITHUB_TOKEN in CI)
|
||||||
|
export GITHUB_TOKEN=$(gh auth token)
|
||||||
|
|
||||||
|
# Search a private catalog added via `specify preset catalog add`
|
||||||
|
specify preset search my-template
|
||||||
|
|
||||||
|
# Install from a private catalog
|
||||||
|
specify preset add my-template
|
||||||
|
```
|
||||||
|
|
||||||
|
The token is attached automatically to requests targeting GitHub domains. Non-GitHub catalog URLs are always fetched without credentials.
|
||||||
|
|
||||||
## Configuration Files
|
## Configuration Files
|
||||||
|
|
||||||
|
|||||||
80
src/specify_cli/_github_http.py
Normal file
80
src/specify_cli/_github_http.py
Normal file
@@ -0,0 +1,80 @@
|
|||||||
|
"""Shared GitHub-authenticated HTTP helpers.
|
||||||
|
|
||||||
|
Used by both ExtensionCatalog and PresetCatalog to attach
|
||||||
|
GITHUB_TOKEN / GH_TOKEN credentials to requests targeting
|
||||||
|
GitHub-hosted domains, while preventing token leakage to
|
||||||
|
third-party hosts on redirects.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import urllib.request
|
||||||
|
from urllib.parse import urlparse
|
||||||
|
from typing import Dict
|
||||||
|
|
||||||
|
# GitHub-owned hostnames that should receive the Authorization header.
|
||||||
|
# Includes codeload.github.com because GitHub archive URL downloads
|
||||||
|
# (e.g. /archive/refs/tags/<tag>.zip) redirect there and require auth
|
||||||
|
# for private repositories.
|
||||||
|
GITHUB_HOSTS = frozenset({
|
||||||
|
"raw.githubusercontent.com",
|
||||||
|
"github.com",
|
||||||
|
"api.github.com",
|
||||||
|
"codeload.github.com",
|
||||||
|
})
|
||||||
|
|
||||||
|
|
||||||
|
def build_github_request(url: str) -> urllib.request.Request:
|
||||||
|
"""Build a urllib Request, adding a GitHub auth header when available.
|
||||||
|
|
||||||
|
Reads GITHUB_TOKEN or GH_TOKEN from the environment and attaches an
|
||||||
|
``Authorization: Bearer <value>`` header when the target hostname is one
|
||||||
|
of the known GitHub-owned domains. Non-GitHub URLs are returned as plain
|
||||||
|
requests so credentials are never leaked to third-party hosts.
|
||||||
|
"""
|
||||||
|
headers: Dict[str, str] = {}
|
||||||
|
github_token = (os.environ.get("GITHUB_TOKEN") or "").strip()
|
||||||
|
gh_token = (os.environ.get("GH_TOKEN") or "").strip()
|
||||||
|
token = github_token or gh_token or None
|
||||||
|
hostname = (urlparse(url).hostname or "").lower()
|
||||||
|
if token and hostname in GITHUB_HOSTS:
|
||||||
|
headers["Authorization"] = f"Bearer {token}"
|
||||||
|
return urllib.request.Request(url, headers=headers)
|
||||||
|
|
||||||
|
|
||||||
|
class _StripAuthOnRedirect(urllib.request.HTTPRedirectHandler):
|
||||||
|
"""Redirect handler that drops the Authorization header when leaving GitHub.
|
||||||
|
|
||||||
|
Prevents token leakage to CDNs or other third-party hosts that GitHub
|
||||||
|
may redirect to (e.g. S3 for release asset downloads, objects.githubusercontent.com).
|
||||||
|
Auth is preserved as long as the redirect target remains within GITHUB_HOSTS.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def redirect_request(self, req, fp, code, msg, headers, newurl):
|
||||||
|
original_auth = req.get_header("Authorization")
|
||||||
|
new_req = super().redirect_request(req, fp, code, msg, headers, newurl)
|
||||||
|
if new_req is not None:
|
||||||
|
hostname = (urlparse(newurl).hostname or "").lower()
|
||||||
|
if hostname in GITHUB_HOSTS:
|
||||||
|
if original_auth:
|
||||||
|
new_req.add_unredirected_header("Authorization", original_auth)
|
||||||
|
else:
|
||||||
|
new_req.headers.pop("Authorization", None)
|
||||||
|
new_req.unredirected_hdrs.pop("Authorization", None)
|
||||||
|
return new_req
|
||||||
|
|
||||||
|
|
||||||
|
def open_github_url(url: str, timeout: int = 10):
|
||||||
|
"""Open a URL with GitHub auth, stripping the header on cross-host redirects.
|
||||||
|
|
||||||
|
When the request carries an Authorization header, a custom redirect
|
||||||
|
handler drops that header if the redirect target is not a GitHub-owned
|
||||||
|
domain, preventing token leakage to CDNs or other third-party hosts
|
||||||
|
that GitHub may redirect to (e.g. S3 for release asset downloads).
|
||||||
|
"""
|
||||||
|
req = build_github_request(url)
|
||||||
|
|
||||||
|
if not req.get_header("Authorization"):
|
||||||
|
return urllib.request.urlopen(req, timeout=timeout)
|
||||||
|
|
||||||
|
opener = urllib.request.build_opener(_StripAuthOnRedirect)
|
||||||
|
return opener.open(req, timeout=timeout)
|
||||||
@@ -1539,6 +1539,22 @@ class ExtensionCatalog:
|
|||||||
if not parsed.netloc:
|
if not parsed.netloc:
|
||||||
raise ValidationError("Catalog URL must be a valid URL with a host.")
|
raise ValidationError("Catalog URL must be a valid URL with a host.")
|
||||||
|
|
||||||
|
def _make_request(self, url: str):
|
||||||
|
"""Build a urllib Request, adding a GitHub auth header when available.
|
||||||
|
|
||||||
|
Delegates to :func:`specify_cli._github_http.build_github_request`.
|
||||||
|
"""
|
||||||
|
from specify_cli._github_http import build_github_request
|
||||||
|
return build_github_request(url)
|
||||||
|
|
||||||
|
def _open_url(self, url: str, timeout: int = 10):
|
||||||
|
"""Open a URL with GitHub auth, stripping the header on cross-host redirects.
|
||||||
|
|
||||||
|
Delegates to :func:`specify_cli._github_http.open_github_url`.
|
||||||
|
"""
|
||||||
|
from specify_cli._github_http import open_github_url
|
||||||
|
return open_github_url(url, timeout)
|
||||||
|
|
||||||
def _load_catalog_config(self, config_path: Path) -> Optional[List[CatalogEntry]]:
|
def _load_catalog_config(self, config_path: Path) -> Optional[List[CatalogEntry]]:
|
||||||
"""Load catalog stack configuration from a YAML file.
|
"""Load catalog stack configuration from a YAML file.
|
||||||
|
|
||||||
@@ -1695,7 +1711,6 @@ class ExtensionCatalog:
|
|||||||
Raises:
|
Raises:
|
||||||
ExtensionError: If catalog cannot be fetched or has invalid format
|
ExtensionError: If catalog cannot be fetched or has invalid format
|
||||||
"""
|
"""
|
||||||
import urllib.request
|
|
||||||
import urllib.error
|
import urllib.error
|
||||||
|
|
||||||
# Determine cache file paths (backward compat for default catalog)
|
# Determine cache file paths (backward compat for default catalog)
|
||||||
@@ -1729,7 +1744,7 @@ class ExtensionCatalog:
|
|||||||
|
|
||||||
# Fetch from network
|
# Fetch from network
|
||||||
try:
|
try:
|
||||||
with urllib.request.urlopen(entry.url, timeout=10) as response:
|
with self._open_url(entry.url, timeout=10) as response:
|
||||||
catalog_data = json.loads(response.read())
|
catalog_data = json.loads(response.read())
|
||||||
|
|
||||||
if "schema_version" not in catalog_data or "extensions" not in catalog_data:
|
if "schema_version" not in catalog_data or "extensions" not in catalog_data:
|
||||||
@@ -1843,10 +1858,9 @@ class ExtensionCatalog:
|
|||||||
catalog_url = self.get_catalog_url()
|
catalog_url = self.get_catalog_url()
|
||||||
|
|
||||||
try:
|
try:
|
||||||
import urllib.request
|
|
||||||
import urllib.error
|
import urllib.error
|
||||||
|
|
||||||
with urllib.request.urlopen(catalog_url, timeout=10) as response:
|
with self._open_url(catalog_url, timeout=10) as response:
|
||||||
catalog_data = json.loads(response.read())
|
catalog_data = json.loads(response.read())
|
||||||
|
|
||||||
# Validate catalog structure
|
# Validate catalog structure
|
||||||
@@ -1957,7 +1971,6 @@ class ExtensionCatalog:
|
|||||||
Raises:
|
Raises:
|
||||||
ExtensionError: If extension not found or download fails
|
ExtensionError: If extension not found or download fails
|
||||||
"""
|
"""
|
||||||
import urllib.request
|
|
||||||
import urllib.error
|
import urllib.error
|
||||||
|
|
||||||
# Get extension info from catalog
|
# Get extension info from catalog
|
||||||
@@ -1997,7 +2010,7 @@ class ExtensionCatalog:
|
|||||||
|
|
||||||
# Download the ZIP file
|
# Download the ZIP file
|
||||||
try:
|
try:
|
||||||
with urllib.request.urlopen(download_url, timeout=60) as response:
|
with self._open_url(download_url, timeout=60) as response:
|
||||||
zip_data = response.read()
|
zip_data = response.read()
|
||||||
|
|
||||||
zip_path.write_bytes(zip_data)
|
zip_path.write_bytes(zip_data)
|
||||||
|
|||||||
@@ -1831,6 +1831,22 @@ class PresetCatalog:
|
|||||||
"Catalog URL must be a valid URL with a host."
|
"Catalog URL must be a valid URL with a host."
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def _make_request(self, url: str):
|
||||||
|
"""Build a urllib Request, adding a GitHub auth header when available.
|
||||||
|
|
||||||
|
Delegates to :func:`specify_cli._github_http.build_github_request`.
|
||||||
|
"""
|
||||||
|
from specify_cli._github_http import build_github_request
|
||||||
|
return build_github_request(url)
|
||||||
|
|
||||||
|
def _open_url(self, url: str, timeout: int = 10):
|
||||||
|
"""Open a URL with GitHub auth, stripping the header on cross-host redirects.
|
||||||
|
|
||||||
|
Delegates to :func:`specify_cli._github_http.open_github_url`.
|
||||||
|
"""
|
||||||
|
from specify_cli._github_http import open_github_url
|
||||||
|
return open_github_url(url, timeout)
|
||||||
|
|
||||||
def _load_catalog_config(self, config_path: Path) -> Optional[List[PresetCatalogEntry]]:
|
def _load_catalog_config(self, config_path: Path) -> Optional[List[PresetCatalogEntry]]:
|
||||||
"""Load catalog stack configuration from a YAML file.
|
"""Load catalog stack configuration from a YAML file.
|
||||||
|
|
||||||
@@ -2013,10 +2029,7 @@ class PresetCatalog:
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
try:
|
try:
|
||||||
import urllib.request
|
with self._open_url(entry.url, timeout=10) as response:
|
||||||
import urllib.error
|
|
||||||
|
|
||||||
with urllib.request.urlopen(entry.url, timeout=10) as response:
|
|
||||||
catalog_data = json.loads(response.read())
|
catalog_data = json.loads(response.read())
|
||||||
|
|
||||||
if (
|
if (
|
||||||
@@ -2109,10 +2122,7 @@ class PresetCatalog:
|
|||||||
pass
|
pass
|
||||||
|
|
||||||
try:
|
try:
|
||||||
import urllib.request
|
with self._open_url(catalog_url, timeout=10) as response:
|
||||||
import urllib.error
|
|
||||||
|
|
||||||
with urllib.request.urlopen(catalog_url, timeout=10) as response:
|
|
||||||
catalog_data = json.loads(response.read())
|
catalog_data = json.loads(response.read())
|
||||||
|
|
||||||
if (
|
if (
|
||||||
@@ -2231,7 +2241,6 @@ class PresetCatalog:
|
|||||||
Raises:
|
Raises:
|
||||||
PresetError: If pack not found or download fails
|
PresetError: If pack not found or download fails
|
||||||
"""
|
"""
|
||||||
import urllib.request
|
|
||||||
import urllib.error
|
import urllib.error
|
||||||
|
|
||||||
pack_info = self.get_pack_info(pack_id)
|
pack_info = self.get_pack_info(pack_id)
|
||||||
@@ -2283,7 +2292,7 @@ class PresetCatalog:
|
|||||||
zip_path = target_dir / zip_filename
|
zip_path = target_dir / zip_filename
|
||||||
|
|
||||||
try:
|
try:
|
||||||
with urllib.request.urlopen(download_url, timeout=60) as response:
|
with self._open_url(download_url, timeout=60) as response:
|
||||||
zip_data = response.read()
|
zip_data = response.read()
|
||||||
|
|
||||||
zip_path.write_bytes(zip_data)
|
zip_path.write_bytes(zip_data)
|
||||||
|
|||||||
@@ -2416,6 +2416,215 @@ class TestExtensionCatalog:
|
|||||||
assert not catalog.cache_file.exists()
|
assert not catalog.cache_file.exists()
|
||||||
assert not catalog.cache_metadata_file.exists()
|
assert not catalog.cache_metadata_file.exists()
|
||||||
|
|
||||||
|
# --- _make_request / GitHub auth ---
|
||||||
|
|
||||||
|
def _make_catalog(self, temp_dir):
|
||||||
|
project_dir = temp_dir / "project"
|
||||||
|
project_dir.mkdir()
|
||||||
|
(project_dir / ".specify").mkdir()
|
||||||
|
return ExtensionCatalog(project_dir)
|
||||||
|
|
||||||
|
def test_make_request_no_token_no_auth_header(self, temp_dir, monkeypatch):
|
||||||
|
"""Without a token, requests carry no Authorization header."""
|
||||||
|
monkeypatch.delenv("GITHUB_TOKEN", raising=False)
|
||||||
|
monkeypatch.delenv("GH_TOKEN", raising=False)
|
||||||
|
catalog = self._make_catalog(temp_dir)
|
||||||
|
req = catalog._make_request("https://raw.githubusercontent.com/org/repo/main/catalog.json")
|
||||||
|
assert "Authorization" not in req.headers
|
||||||
|
|
||||||
|
def test_make_request_whitespace_only_github_token_ignored(self, temp_dir, monkeypatch):
|
||||||
|
"""A whitespace-only GITHUB_TOKEN is treated as unset."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", " ")
|
||||||
|
monkeypatch.delenv("GH_TOKEN", raising=False)
|
||||||
|
catalog = self._make_catalog(temp_dir)
|
||||||
|
req = catalog._make_request("https://raw.githubusercontent.com/org/repo/main/catalog.json")
|
||||||
|
assert "Authorization" not in req.headers
|
||||||
|
|
||||||
|
def test_make_request_whitespace_github_token_falls_back_to_gh_token(self, temp_dir, monkeypatch):
|
||||||
|
"""When GITHUB_TOKEN is whitespace-only, GH_TOKEN is used as fallback."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", " ")
|
||||||
|
monkeypatch.setenv("GH_TOKEN", "ghp_fallback")
|
||||||
|
catalog = self._make_catalog(temp_dir)
|
||||||
|
req = catalog._make_request("https://raw.githubusercontent.com/org/repo/main/catalog.json")
|
||||||
|
assert req.get_header("Authorization") == "Bearer ghp_fallback"
|
||||||
|
|
||||||
|
def test_make_request_github_token_added_for_raw_githubusercontent(self, temp_dir, monkeypatch):
|
||||||
|
"""GITHUB_TOKEN is attached for raw.githubusercontent.com URLs."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
monkeypatch.delenv("GH_TOKEN", raising=False)
|
||||||
|
catalog = self._make_catalog(temp_dir)
|
||||||
|
req = catalog._make_request("https://raw.githubusercontent.com/org/repo/main/catalog.json")
|
||||||
|
assert req.get_header("Authorization") == "Bearer ghp_testtoken"
|
||||||
|
|
||||||
|
def test_make_request_gh_token_fallback(self, temp_dir, monkeypatch):
|
||||||
|
"""GH_TOKEN is used when GITHUB_TOKEN is absent."""
|
||||||
|
monkeypatch.delenv("GITHUB_TOKEN", raising=False)
|
||||||
|
monkeypatch.setenv("GH_TOKEN", "ghp_ghtoken")
|
||||||
|
catalog = self._make_catalog(temp_dir)
|
||||||
|
req = catalog._make_request("https://github.com/org/repo/releases/download/v1/ext.zip")
|
||||||
|
assert req.get_header("Authorization") == "Bearer ghp_ghtoken"
|
||||||
|
|
||||||
|
def test_make_request_github_token_takes_precedence_over_gh_token(self, temp_dir, monkeypatch):
|
||||||
|
"""GITHUB_TOKEN takes precedence over GH_TOKEN when both are set."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_primary")
|
||||||
|
monkeypatch.setenv("GH_TOKEN", "ghp_secondary")
|
||||||
|
catalog = self._make_catalog(temp_dir)
|
||||||
|
req = catalog._make_request("https://api.github.com/repos/org/repo")
|
||||||
|
assert req.get_header("Authorization") == "Bearer ghp_primary"
|
||||||
|
|
||||||
|
def test_make_request_token_not_added_for_non_github_url(self, temp_dir, monkeypatch):
|
||||||
|
"""Auth header is never attached to non-GitHub URLs to prevent credential leakage."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
catalog = self._make_catalog(temp_dir)
|
||||||
|
req = catalog._make_request("https://internal.example.com/catalog.json")
|
||||||
|
assert "Authorization" not in req.headers
|
||||||
|
|
||||||
|
def test_make_request_token_not_added_for_github_lookalike_host(self, temp_dir, monkeypatch):
|
||||||
|
"""Auth header is not attached to hosts that include github.com as a suffix."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
catalog = self._make_catalog(temp_dir)
|
||||||
|
req = catalog._make_request("https://github.com.evil.com/org/repo/releases/download/v1/ext.zip")
|
||||||
|
assert "Authorization" not in req.headers
|
||||||
|
|
||||||
|
def test_make_request_token_not_added_for_github_in_path(self, temp_dir, monkeypatch):
|
||||||
|
"""Auth header is not attached when github.com appears only in the URL path."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
catalog = self._make_catalog(temp_dir)
|
||||||
|
req = catalog._make_request("https://evil.example.com/github.com/org/repo/releases/download/v1/ext.zip")
|
||||||
|
assert "Authorization" not in req.headers
|
||||||
|
|
||||||
|
def test_make_request_token_not_added_for_github_in_query(self, temp_dir, monkeypatch):
|
||||||
|
"""Auth header is not attached when github.com appears only in the query string."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
catalog = self._make_catalog(temp_dir)
|
||||||
|
req = catalog._make_request("https://evil.example.com/download?source=https://github.com/org/repo/v1/ext.zip")
|
||||||
|
assert "Authorization" not in req.headers
|
||||||
|
|
||||||
|
def test_make_request_token_added_for_api_github_com(self, temp_dir, monkeypatch):
|
||||||
|
"""GITHUB_TOKEN is attached for api.github.com URLs."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
catalog = self._make_catalog(temp_dir)
|
||||||
|
req = catalog._make_request("https://api.github.com/repos/org/repo/releases/assets/1")
|
||||||
|
assert req.get_header("Authorization") == "Bearer ghp_testtoken"
|
||||||
|
|
||||||
|
def test_make_request_token_added_for_codeload_github_com(self, temp_dir, monkeypatch):
|
||||||
|
"""GITHUB_TOKEN is attached for codeload.github.com URLs (GitHub archive redirects)."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
catalog = self._make_catalog(temp_dir)
|
||||||
|
req = catalog._make_request("https://codeload.github.com/org/repo/zip/refs/tags/v1.0.0")
|
||||||
|
assert req.get_header("Authorization") == "Bearer ghp_testtoken"
|
||||||
|
|
||||||
|
def test_redirect_preserves_auth_for_github_to_codeload(self):
|
||||||
|
"""Auth header is preserved when GitHub redirects to codeload.github.com."""
|
||||||
|
from specify_cli._github_http import _StripAuthOnRedirect
|
||||||
|
from urllib.request import Request
|
||||||
|
import io
|
||||||
|
|
||||||
|
handler = _StripAuthOnRedirect()
|
||||||
|
original_url = "https://github.com/org/repo/archive/refs/tags/v1.zip"
|
||||||
|
redirect_url = "https://codeload.github.com/org/repo/zip/refs/tags/v1"
|
||||||
|
req = Request(original_url, headers={"Authorization": "Bearer ghp_test"})
|
||||||
|
fp = io.BytesIO(b"")
|
||||||
|
new_req = handler.redirect_request(req, fp, 302, "Found", {}, redirect_url)
|
||||||
|
assert new_req is not None
|
||||||
|
auth = new_req.get_header("Authorization") or new_req.unredirected_hdrs.get("Authorization")
|
||||||
|
assert auth == "Bearer ghp_test"
|
||||||
|
|
||||||
|
def test_redirect_strips_auth_for_github_to_external(self):
|
||||||
|
"""Auth header is stripped when GitHub redirects to a non-GitHub host."""
|
||||||
|
from specify_cli._github_http import _StripAuthOnRedirect
|
||||||
|
from urllib.request import Request
|
||||||
|
import io
|
||||||
|
|
||||||
|
handler = _StripAuthOnRedirect()
|
||||||
|
original_url = "https://github.com/org/repo/releases/download/v1/asset.zip"
|
||||||
|
redirect_url = "https://objects.githubusercontent.com/github-production-release-asset/12345"
|
||||||
|
req = Request(original_url, headers={"Authorization": "Bearer ghp_test"})
|
||||||
|
fp = io.BytesIO(b"")
|
||||||
|
new_req = handler.redirect_request(req, fp, 302, "Found", {}, redirect_url)
|
||||||
|
assert new_req is not None
|
||||||
|
auth_header = new_req.headers.get("Authorization")
|
||||||
|
auth_unredirected = new_req.unredirected_hdrs.get("Authorization")
|
||||||
|
assert auth_header is None
|
||||||
|
assert auth_unredirected is None
|
||||||
|
|
||||||
|
def test_fetch_single_catalog_sends_auth_header(self, temp_dir, monkeypatch):
|
||||||
|
"""_fetch_single_catalog passes Authorization header via opener for GitHub URLs."""
|
||||||
|
from unittest.mock import patch, MagicMock
|
||||||
|
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
catalog = self._make_catalog(temp_dir)
|
||||||
|
|
||||||
|
catalog_data = {"schema_version": "1.0", "extensions": {}}
|
||||||
|
mock_response = MagicMock()
|
||||||
|
mock_response.read.return_value = json.dumps(catalog_data).encode()
|
||||||
|
mock_response.__enter__ = lambda s: s
|
||||||
|
mock_response.__exit__ = MagicMock(return_value=False)
|
||||||
|
|
||||||
|
captured = {}
|
||||||
|
mock_opener = MagicMock()
|
||||||
|
|
||||||
|
def fake_open(req, timeout=None):
|
||||||
|
captured["req"] = req
|
||||||
|
return mock_response
|
||||||
|
|
||||||
|
mock_opener.open.side_effect = fake_open
|
||||||
|
|
||||||
|
entry = CatalogEntry(
|
||||||
|
url="https://raw.githubusercontent.com/org/repo/main/catalog.json",
|
||||||
|
name="private",
|
||||||
|
priority=1,
|
||||||
|
install_allowed=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
with patch("urllib.request.build_opener", return_value=mock_opener):
|
||||||
|
catalog._fetch_single_catalog(entry, force_refresh=True)
|
||||||
|
|
||||||
|
assert captured["req"].get_header("Authorization") == "Bearer ghp_testtoken"
|
||||||
|
|
||||||
|
def test_download_extension_sends_auth_header(self, temp_dir, monkeypatch):
|
||||||
|
"""download_extension passes Authorization header via opener for GitHub URLs."""
|
||||||
|
from unittest.mock import patch, MagicMock
|
||||||
|
import zipfile, io
|
||||||
|
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
catalog = self._make_catalog(temp_dir)
|
||||||
|
|
||||||
|
# Build a minimal valid ZIP in memory
|
||||||
|
zip_buf = io.BytesIO()
|
||||||
|
with zipfile.ZipFile(zip_buf, "w") as zf:
|
||||||
|
zf.writestr("extension.yml", "id: test-ext\nname: Test\nversion: 1.0.0\n")
|
||||||
|
zip_bytes = zip_buf.getvalue()
|
||||||
|
|
||||||
|
mock_response = MagicMock()
|
||||||
|
mock_response.read.return_value = zip_bytes
|
||||||
|
mock_response.__enter__ = lambda s: s
|
||||||
|
mock_response.__exit__ = MagicMock(return_value=False)
|
||||||
|
|
||||||
|
captured = {}
|
||||||
|
|
||||||
|
mock_opener = MagicMock()
|
||||||
|
|
||||||
|
def fake_open(req, timeout=None):
|
||||||
|
captured["req"] = req
|
||||||
|
return mock_response
|
||||||
|
|
||||||
|
mock_opener.open.side_effect = fake_open
|
||||||
|
|
||||||
|
ext_info = {
|
||||||
|
"id": "test-ext",
|
||||||
|
"name": "Test Extension",
|
||||||
|
"version": "1.0.0",
|
||||||
|
"download_url": "https://github.com/org/repo/releases/download/v1/test-ext.zip",
|
||||||
|
}
|
||||||
|
|
||||||
|
with patch.object(catalog, "get_extension_info", return_value=ext_info), \
|
||||||
|
patch("urllib.request.build_opener", return_value=mock_opener):
|
||||||
|
catalog.download_extension("test-ext", target_dir=temp_dir)
|
||||||
|
|
||||||
|
assert captured["req"].get_header("Authorization") == "Bearer ghp_testtoken"
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# ===== CatalogEntry Tests =====
|
# ===== CatalogEntry Tests =====
|
||||||
|
|
||||||
|
|||||||
@@ -1363,6 +1363,166 @@ class TestPresetCatalog:
|
|||||||
catalog = PresetCatalog(project_dir)
|
catalog = PresetCatalog(project_dir)
|
||||||
assert catalog.get_catalog_url() == "https://custom.example.com/catalog.json"
|
assert catalog.get_catalog_url() == "https://custom.example.com/catalog.json"
|
||||||
|
|
||||||
|
# --- _make_request / GitHub auth ---
|
||||||
|
|
||||||
|
def test_make_request_no_token_no_auth_header(self, project_dir, monkeypatch):
|
||||||
|
"""Without a token, requests carry no Authorization header."""
|
||||||
|
monkeypatch.delenv("GITHUB_TOKEN", raising=False)
|
||||||
|
monkeypatch.delenv("GH_TOKEN", raising=False)
|
||||||
|
catalog = PresetCatalog(project_dir)
|
||||||
|
req = catalog._make_request("https://raw.githubusercontent.com/org/repo/main/catalog.json")
|
||||||
|
assert "Authorization" not in req.headers
|
||||||
|
|
||||||
|
def test_make_request_whitespace_only_github_token_ignored(self, project_dir, monkeypatch):
|
||||||
|
"""A whitespace-only GITHUB_TOKEN is treated as unset."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", " ")
|
||||||
|
monkeypatch.delenv("GH_TOKEN", raising=False)
|
||||||
|
catalog = PresetCatalog(project_dir)
|
||||||
|
req = catalog._make_request("https://raw.githubusercontent.com/org/repo/main/catalog.json")
|
||||||
|
assert "Authorization" not in req.headers
|
||||||
|
|
||||||
|
def test_make_request_whitespace_github_token_falls_back_to_gh_token(self, project_dir, monkeypatch):
|
||||||
|
"""When GITHUB_TOKEN is whitespace-only, GH_TOKEN is used as fallback."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", " ")
|
||||||
|
monkeypatch.setenv("GH_TOKEN", "ghp_fallback")
|
||||||
|
catalog = PresetCatalog(project_dir)
|
||||||
|
req = catalog._make_request("https://raw.githubusercontent.com/org/repo/main/catalog.json")
|
||||||
|
assert req.get_header("Authorization") == "Bearer ghp_fallback"
|
||||||
|
|
||||||
|
def test_make_request_github_token_added_for_github_url(self, project_dir, monkeypatch):
|
||||||
|
"""GITHUB_TOKEN is attached for raw.githubusercontent.com URLs."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
monkeypatch.delenv("GH_TOKEN", raising=False)
|
||||||
|
catalog = PresetCatalog(project_dir)
|
||||||
|
req = catalog._make_request("https://raw.githubusercontent.com/org/repo/main/catalog.json")
|
||||||
|
assert req.get_header("Authorization") == "Bearer ghp_testtoken"
|
||||||
|
|
||||||
|
def test_make_request_gh_token_fallback(self, project_dir, monkeypatch):
|
||||||
|
"""GH_TOKEN is used when GITHUB_TOKEN is absent."""
|
||||||
|
monkeypatch.delenv("GITHUB_TOKEN", raising=False)
|
||||||
|
monkeypatch.setenv("GH_TOKEN", "ghp_ghtoken")
|
||||||
|
catalog = PresetCatalog(project_dir)
|
||||||
|
req = catalog._make_request("https://github.com/org/repo/releases/download/v1/pack.zip")
|
||||||
|
assert req.get_header("Authorization") == "Bearer ghp_ghtoken"
|
||||||
|
|
||||||
|
def test_make_request_github_token_takes_precedence(self, project_dir, monkeypatch):
|
||||||
|
"""GITHUB_TOKEN takes precedence over GH_TOKEN when both are set."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_primary")
|
||||||
|
monkeypatch.setenv("GH_TOKEN", "ghp_secondary")
|
||||||
|
catalog = PresetCatalog(project_dir)
|
||||||
|
req = catalog._make_request("https://api.github.com/repos/org/repo")
|
||||||
|
assert req.get_header("Authorization") == "Bearer ghp_primary"
|
||||||
|
|
||||||
|
def test_make_request_token_added_for_codeload_github_com(self, project_dir, monkeypatch):
|
||||||
|
"""GITHUB_TOKEN is attached for codeload.github.com URLs (GitHub archive redirects)."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
catalog = PresetCatalog(project_dir)
|
||||||
|
req = catalog._make_request("https://codeload.github.com/org/repo/zip/refs/tags/v1.0.0")
|
||||||
|
assert req.get_header("Authorization") == "Bearer ghp_testtoken"
|
||||||
|
|
||||||
|
def test_make_request_token_not_added_for_non_github_url(self, project_dir, monkeypatch):
|
||||||
|
"""Auth header is never attached to non-GitHub URLs to prevent credential leakage."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
catalog = PresetCatalog(project_dir)
|
||||||
|
req = catalog._make_request("https://internal.example.com/catalog.json")
|
||||||
|
assert "Authorization" not in req.headers
|
||||||
|
|
||||||
|
def test_make_request_token_not_added_for_github_lookalike_host(self, project_dir, monkeypatch):
|
||||||
|
"""Auth header is not attached to hosts that include github.com as a suffix."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
catalog = PresetCatalog(project_dir)
|
||||||
|
req = catalog._make_request("https://github.com.evil.com/org/repo/releases/download/v1/pack.zip")
|
||||||
|
assert "Authorization" not in req.headers
|
||||||
|
|
||||||
|
def test_make_request_token_not_added_for_github_in_path(self, project_dir, monkeypatch):
|
||||||
|
"""Auth header is not attached when github.com appears only in the URL path."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
catalog = PresetCatalog(project_dir)
|
||||||
|
req = catalog._make_request("https://evil.example.com/github.com/org/repo/releases/download/v1/pack.zip")
|
||||||
|
assert "Authorization" not in req.headers
|
||||||
|
|
||||||
|
def test_make_request_token_not_added_for_github_in_query(self, project_dir, monkeypatch):
|
||||||
|
"""Auth header is not attached when github.com appears only in the query string."""
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
catalog = PresetCatalog(project_dir)
|
||||||
|
req = catalog._make_request("https://evil.example.com/download?source=https://github.com/org/repo/v1/pack.zip")
|
||||||
|
assert "Authorization" not in req.headers
|
||||||
|
|
||||||
|
def test_fetch_single_catalog_sends_auth_header(self, project_dir, monkeypatch):
|
||||||
|
"""_fetch_single_catalog passes Authorization header via opener for GitHub URLs."""
|
||||||
|
from unittest.mock import patch, MagicMock
|
||||||
|
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
catalog = PresetCatalog(project_dir)
|
||||||
|
|
||||||
|
catalog_data = {"schema_version": "1.0", "presets": {}}
|
||||||
|
mock_response = MagicMock()
|
||||||
|
mock_response.read.return_value = json.dumps(catalog_data).encode()
|
||||||
|
mock_response.__enter__ = lambda s: s
|
||||||
|
mock_response.__exit__ = MagicMock(return_value=False)
|
||||||
|
|
||||||
|
captured = {}
|
||||||
|
mock_opener = MagicMock()
|
||||||
|
|
||||||
|
def fake_open(req, timeout=None):
|
||||||
|
captured["req"] = req
|
||||||
|
return mock_response
|
||||||
|
|
||||||
|
mock_opener.open.side_effect = fake_open
|
||||||
|
|
||||||
|
entry = PresetCatalogEntry(
|
||||||
|
url="https://raw.githubusercontent.com/org/repo/main/presets/catalog.json",
|
||||||
|
name="private",
|
||||||
|
priority=1,
|
||||||
|
install_allowed=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
with patch("urllib.request.build_opener", return_value=mock_opener):
|
||||||
|
catalog._fetch_single_catalog(entry, force_refresh=True)
|
||||||
|
|
||||||
|
assert captured["req"].get_header("Authorization") == "Bearer ghp_testtoken"
|
||||||
|
|
||||||
|
def test_download_pack_sends_auth_header(self, project_dir, monkeypatch):
|
||||||
|
"""download_pack passes Authorization header via opener for GitHub URLs."""
|
||||||
|
from unittest.mock import patch, MagicMock
|
||||||
|
|
||||||
|
monkeypatch.setenv("GITHUB_TOKEN", "ghp_testtoken")
|
||||||
|
catalog = PresetCatalog(project_dir)
|
||||||
|
|
||||||
|
import io
|
||||||
|
zip_buf = io.BytesIO()
|
||||||
|
with zipfile.ZipFile(zip_buf, "w") as zf:
|
||||||
|
zf.writestr("preset.yml", "id: test-pack\nname: Test\nversion: 1.0.0\n")
|
||||||
|
zip_bytes = zip_buf.getvalue()
|
||||||
|
|
||||||
|
mock_response = MagicMock()
|
||||||
|
mock_response.read.return_value = zip_bytes
|
||||||
|
mock_response.__enter__ = lambda s: s
|
||||||
|
mock_response.__exit__ = MagicMock(return_value=False)
|
||||||
|
|
||||||
|
captured = {}
|
||||||
|
mock_opener = MagicMock()
|
||||||
|
|
||||||
|
def fake_open(req, timeout=None):
|
||||||
|
captured["req"] = req
|
||||||
|
return mock_response
|
||||||
|
|
||||||
|
mock_opener.open.side_effect = fake_open
|
||||||
|
|
||||||
|
pack_info = {
|
||||||
|
"id": "test-pack",
|
||||||
|
"name": "Test Pack",
|
||||||
|
"version": "1.0.0",
|
||||||
|
"download_url": "https://github.com/org/repo/releases/download/v1/test-pack.zip",
|
||||||
|
"_install_allowed": True,
|
||||||
|
}
|
||||||
|
|
||||||
|
with patch.object(catalog, "get_pack_info", return_value=pack_info), \
|
||||||
|
patch("urllib.request.build_opener", return_value=mock_opener):
|
||||||
|
catalog.download_pack("test-pack", target_dir=project_dir)
|
||||||
|
|
||||||
|
assert captured["req"].get_header("Authorization") == "Bearer ghp_testtoken"
|
||||||
|
|
||||||
|
|
||||||
# ===== Integration Tests =====
|
# ===== Integration Tests =====
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user