viewengine/viewengine.client
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
8faf3d1ae7
viewengine.client/PROJECT_SUMMARY.md

7.6 KiB
Raw Blame History

ViewEngine.Client - Project Summary

Overview

The ViewEngine.Client is a production-ready .NET 9 NuGet package that provides a clean, type-safe client library for consuming the ViewEngine REST API. It enables developers to easily retrieve web pages, extract content, and manage distributed web scraping operations.

Project Structure

ViewEngine.Client/
├── Configuration/
│   └── ViewEngineOptions.cs           # Configuration options for the client
├── Extensions/
│   └── ServiceCollectionExtensions.cs # Dependency injection helpers
├── Models/
│   ├── ClientManagementModels.cs      # DTOs for client management API
│   └── RetrievalModels.cs             # DTOs for retrieval API
├── ViewEngineClient.cs                # Main client implementation
├── ViewEngine.Client.csproj           # Project file with NuGet metadata
├── README.md                          # Comprehensive documentation
├── QUICKSTART.md                      # Quick reference guide
├── Examples.md                        # Detailed usage examples
└── PROJECT_SUMMARY.md                 # This file

Key Features

Core Functionality

  • ✅ Submit web page retrieval requests via MCP API
  • ✅ Poll for job status and retrieve results
  • ✅ Download decrypted page content with full text extraction
  • ✅ Access screenshots, thumbnails, and artifacts
  • ✅ Manage clients and route jobs to specific feeders
  • ✅ Built-in retry logic with exponential backoff
  • ✅ Full support for API key authentication
  • ✅ Type-safe request/response models
  • ✅ Async/await support throughout
  • ✅ Dependency injection integration

Advanced Features

  • ✅ Rate limit handling with retry-after support
  • ✅ Exponential backoff for transient failures
  • ✅ Server error retry logic
  • ✅ Platform-specific retrieval (Android, iOS, Windows)
  • ✅ Priority-based job scheduling
  • ✅ Client routing with ClientId or CustomUserId
  • ✅ Force refresh option to bypass cache
  • ✅ Configurable timeouts and polling intervals

Client Management

  • ✅ Add clients programmatically
  • ✅ List and filter clients
  • ✅ Update client settings
  • ✅ Suspend/activate clients
  • ✅ Delete clients
  • ✅ Get detailed client statistics
  • ✅ Monitor feeder online status
  • ✅ Track job processing metrics

API Coverage

MCP Retrieval API (/mcp/*)

  • POST /mcp/retrieve - Submit retrieval request
  • GET /mcp/retrieve/{requestId} - Get retrieval status
  • GET /mcp/retrieve/{requestId}/content - Download page content
  • GET /mcp/tools - List available tools

Client Management API (/v1/clients/*)

  • POST /v1/clients - Add client
  • GET /v1/clients - List clients
  • PUT /v1/clients/{id} - Update client
  • PUT /v1/clients/{id}/suspend - Suspend client
  • PUT /v1/clients/{id}/activate - Activate client
  • DELETE /v1/clients/{id} - Delete client
  • GET /v1/clients/{id}/stats - Get client stats

NuGet Package Details

Package Metadata

  • Package ID: ViewEngine.Client
  • Version: 1.0.0
  • Target Framework: .NET 9.0
  • Authors: David H Friedel Jr
  • License: MIT
  • Output Path: C:\Users\logik\Dropbox\Nugets\ViewEngine.Client.1.0.0.nupkg

Dependencies

  • Microsoft.Extensions.Http (9.0.0)
  • Microsoft.Extensions.Options (9.0.0)
  • Microsoft.Extensions.Options.ConfigurationExtensions (9.0.0)

Package Contents

  • Compiled DLL with XML documentation
  • README.md for NuGet.org package page
  • Full IntelliSense support
  • Source link support (future)

Usage Patterns

1. Simple Usage (No DI)

var client = new ViewEngineClient("ak_your-api-key-here");
var page = await client.RetrieveAndWaitAsync(new SubmitRetrievalRequest
{
    Url = "https://example.com"
});

2. Dependency Injection (Recommended)

// Program.cs
builder.Services.AddViewEngineClient(builder.Configuration);

// Controller
public class MyController : ControllerBase
{
    private readonly ViewEngineClient _client;
    public MyController(ViewEngineClient client) => _client = client;
}

3. Options Pattern

var options = new ViewEngineOptions
{
    ApiKey = "ak_your-api-key-here",
    TimeoutSeconds = 120,
    MaxRetries = 3
};
var client = new ViewEngineClient(options);

Architecture Decisions

1. Multiple Constructors

  • Simple constructor with API key only
  • IOptions constructor for DI
  • HttpClient constructor for advanced DI scenarios
  • Flexible and supports various usage patterns

2. Retry Logic

  • Automatic retry for rate limits (429)
  • Automatic retry for server errors (500-599)
  • Automatic retry for network timeouts
  • Respects Retry-After headers
  • Exponential backoff with configurable base delay

3. Error Handling

  • Throws HttpRequestException for HTTP errors
  • Throws InvalidOperationException for failed retrievals
  • Throws OperationCanceledException for canceled requests
  • All errors include meaningful messages

4. Resource Management

  • Implements IDisposable
  • Properly disposes HttpClient when not injected
  • Does not dispose injected HttpClient (DI container manages it)

5. Type Safety

  • Strongly-typed DTOs for all requests/responses
  • Nullable reference types enabled
  • XML documentation for all public APIs
  • IntelliSense support

Testing Recommendations

Unit Testing

// Mock HttpClient for unit tests
var mockHandler = new MockHttpMessageHandler();
var httpClient = new HttpClient(mockHandler);
var options = Options.Create(new ViewEngineOptions { ApiKey = "test" });
var client = new ViewEngineClient(httpClient, options);

Integration Testing

// Use real API with test API key
var client = new ViewEngineClient("ak_test_key");
var page = await client.RetrieveAndWaitAsync(new SubmitRetrievalRequest
{
    Url = "https://example.com"
});
Assert.NotNull(page.Title);

Build Information

  • Status: ✅ Build succeeded (0 warnings, 0 errors)
  • Package Created: ✅ ViewEngine.Client.1.0.0.nupkg (30,151 bytes)
  • Documentation: ✅ XML documentation generated
  • Target Framework: .NET 9.0
  • Language Version: Latest

Next Steps

For Developers Using This Package

  1. Install via NuGet: dotnet add package ViewEngine.Client
  2. Get API key from https://www.viewengine.io
  3. Follow QUICKSTART.md for immediate use
  4. See Examples.md for advanced scenarios

For Package Maintainers

  1. ✅ Initial release complete
  2. 🔄 Publish to NuGet.org (manual step)
  3. 🔄 Add source link support
  4. 🔄 Add unit tests project
  5. 🔄 Add integration tests
  6. 🔄 Set up CI/CD pipeline
  7. 🔄 Add code coverage reporting

Future Enhancements

  • Add webhook support for job completion notifications
  • Add bulk retrieval helper methods
  • Add caching layer for frequently accessed pages
  • Add telemetry and diagnostics
  • Add support for .NET Standard 2.0 (broader compatibility)
  • Add rate limit awareness (track limits client-side)
  • Add health check endpoint support
  • Add billing/usage API support

Documentation

  • README.md: Main documentation with installation, configuration, and usage
  • QUICKSTART.md: Quick reference for common tasks
  • Examples.md: Comprehensive examples for all scenarios
  • XML Docs: IntelliSense documentation in code
  • API Docs: https://www.viewengine.io/docs

Support

License

MIT License - Copyright © 2025 ViewEngine

Package Created: November 22, 2025 Build Status: ✅ Ready for distribution Quality: Production-ready