ViewEngine.Client

Official .NET client library for consuming the ViewEngine REST API. Retrieve web pages, extract content, and process web data with ViewEngine's distributed web scraping service.

Installation

dotnet add package ViewEngine.Client

Quick Start

Basic Usage

using ViewEngine.Client;
using ViewEngine.Client.Models;

// Create client with API key
var client = new ViewEngineClient("ak_your-api-key-here");

// Submit a retrieval request
var request = new SubmitRetrievalRequest
{
    Url = "https://example.com",
    TimeoutSeconds = 60,
    Priority = 5,
    GenerateSummary = true  // Optional: Generate AI summary
};

// Submit and wait for completion
var pageData = await client.RetrieveAndWaitAsync(request);

Console.WriteLine($"Title: {pageData.Title}");
Console.WriteLine($"Summary: {pageData.Summary}");  // AI-generated summary if requested
Console.WriteLine($"Body: {pageData.Body}");
Console.WriteLine($"Links found: {pageData.Routes.Count}");

Advanced Usage with Polling

// Submit request
var submitResponse = await client.SubmitRetrievalAsync(request);
Console.WriteLine($"Request ID: {submitResponse.RequestId}");

// Poll for status
RetrievalStatusResponse status;
do
{
    status = await client.GetRetrievalStatusAsync(submitResponse.RequestId);
    Console.WriteLine($"Status: {status.Status}");

    if (status.Status == "complete")
    {
        // Download content
        var content = await client.GetPageContentAsync(submitResponse.RequestId);
        Console.WriteLine($"Retrieved: {content.Title}");
        break;
    }

    await Task.Delay(2000); // Wait 2 seconds
} while (status.Status == "queued" || status.Status == "processing");

Dependency Injection

ASP.NET Core / Minimal API

appsettings.json:

{
  "ViewEngine": {
    "ApiKey": "ak_your-api-key-here",
    "BaseUrl": "https://www.viewengine.io/api/v1",
    "TimeoutSeconds": 120,
    "MaxRetries": 3
  }
}

Program.cs:

using ViewEngine.Client.Extensions;

var builder = WebApplication.CreateBuilder(args);

// Add ViewEngine client
builder.Services.AddViewEngineClient(builder.Configuration);

var app = builder.Build();

app.MapGet("/retrieve", async (ViewEngineClient client) =>
{
    var request = new SubmitRetrievalRequest { Url = "https://example.com" };
    var pageData = await client.RetrieveAndWaitAsync(request);
    return Results.Ok(pageData);
});

app.Run();

With Options Pattern

using ViewEngine.Client.Extensions;

builder.Services.AddViewEngineClient(options =>
{
    options.ApiKey = "ak_your-api-key-here";
    options.BaseUrl = "https://www.viewengine.io/api/v1";
    options.TimeoutSeconds = 120;
    options.MaxRetries = 3;
});

Features

Web Page Retrieval

var request = new SubmitRetrievalRequest
{
    Url = "https://example.com",
    TimeoutSeconds = 60,
    ForceRefresh = true,
    Priority = 8,
    PreferredPlatform = "Windows", // Android, iOS, or Windows
    GenerateSummary = true  // Generate AI summary of page content
};

var pageData = await client.RetrieveAndWaitAsync(request);

// Access extracted data
Console.WriteLine($"Title: {pageData.Title}");
Console.WriteLine($"Summary: {pageData.Summary}");  // AI-generated summary
Console.WriteLine($"Description: {pageData.MetaDescription}");
Console.WriteLine($"Body Text: {pageData.Body}");
Console.WriteLine($"Favicon: {pageData.FaviconUrl}");

// Navigation links
foreach (var link in pageData.Routes)
{
    Console.WriteLine($"{link.Text} -> {link.Url}");
}

// Body links with ad detection
foreach (var link in pageData.BodyRoutes)
{
    if (link.IsPotentialAd)
        Console.WriteLine($"[AD] {link.Text} ({link.AdReason})");
    else
        Console.WriteLine($"{link.Text} -> {link.Url}");
}

Client Management

// Add a client
var addRequest = new AddClientRequest
{
    Email = "user@example.com",
    Alias = "Production Feeder",
    CustomUserId = "prod-001",
    DailyMaximum = 1000
};

var newClient = await client.AddClientAsync(addRequest);

// List all clients
var clients = await client.GetClientsAsync();
foreach (var c in clients)
{
    Console.WriteLine($"{c.Alias}: {c.FeederOnline ? "Online" : "Offline"}");
}

// Route job to specific client
var routedRequest = new SubmitRetrievalRequest
{
    Url = "https://example.com",
    ClientId = newClient.Id  // or CustomUserId = "prod-001"
};

// Get client stats
var stats = await client.GetClientStatsAsync(newClient.Id);
Console.WriteLine($"Total jobs: {stats.TotalJobsProcessed}");
Console.WriteLine($"Success rate: {stats.SuccessRate}%");

// Suspend/Activate/Delete
await client.SuspendClientAsync(newClient.Id);
await client.ActivateClientAsync(newClient.Id);
await client.DeleteClientAsync(newClient.Id);

Error Handling

try
{
    var pageData = await client.RetrieveAndWaitAsync(request);
}
catch (HttpRequestException ex)
{
    // HTTP errors (network, server errors)
    Console.WriteLine($"HTTP Error: {ex.Message}");
}
catch (InvalidOperationException ex)
{
    // Failed retrieval or invalid responses
    Console.WriteLine($"Operation Error: {ex.Message}");
}
catch (OperationCanceledException)
{
    // Request was canceled
    Console.WriteLine("Request canceled");
}

Configuration Options

Option	Default	Description
ApiKey	(required)	Your ViewEngine API key
BaseUrl	https://www.viewengine.io/api/v1	API base URL
TimeoutSeconds	120	HTTP request timeout
MaxRetries	3	Maximum retry attempts
BaseDelayMs	1000	Base delay for exponential backoff
DefaultPollingIntervalMs	2000	Default polling interval for status checks

Request Options

SubmitRetrievalRequest

Property	Type	Default	Description
Url	string	(required)	URL to retrieve
TimeoutSeconds	int	60	Max wait time (max: 300)
ForceRefresh	bool	false	Bypass cache
RequiredQuorum	int?	null	Feeders that must agree (1-10, Community mode)
Priority	int	5	Job priority (1-10)
ClientId	Guid?	null	Route to specific client
CustomUserId	string?	null	Route using custom ID
PreferredPlatform	string?	null	"Android", "iOS", or "Windows"
GenerateSummary	bool	false	Generate AI summary of page content

Response Models

PageData

public class PageData
{
    public string Title { get; set; }
    public string Body { get; set; }
    public string? MetaDescription { get; set; }
    public string Url { get; set; }
    public string? FaviconUrl { get; set; }
    public string? Thumbnail { get; set; }  // Base64 PNG
    public string? Summary { get; set; }  // AI-generated summary (if requested)
    public List<LinkInfo> Routes { get; set; }
    public List<LinkInfo> BodyRoutes { get; set; }
}

LinkInfo

public class LinkInfo
{
    public string Url { get; set; }
    public string Text { get; set; }
    public int Rank { get; set; }
    public int Occurrences { get; set; }
    public bool IsPotentialAd { get; set; }
    public string? AdReason { get; set; }
}

Retry & Rate Limiting

The client automatically handles:

• Rate limits (429): Respects Retry-After header with exponential backoff
• Server errors (500-5xx): Retries with exponential backoff
• Network timeouts: Retries with exponential backoff
• Configurable retries: Set MaxRetries in options

API Documentation

For complete API documentation, visit: https://www.viewengine.io/docs

Support

• Documentation: https://www.viewengine.io/docs
• Issues: https://git.marketally.com/viewengine/ViewEngine.Client/issues
• Email: support@marketally.com

License

MIT License - Copyright © 2025 MarketAlly