# SqrtSpace SpaceTime Laravel Sample Application This sample demonstrates how to integrate SqrtSpace SpaceTime with a Laravel application to build memory-efficient, scalable web applications. ## Features Demonstrated ### 1. **Large Dataset API Endpoints** - Streaming JSON responses for large datasets - Paginated queries with automatic memory management - CSV export without memory bloat ### 2. **Background Job Processing** - Memory-aware queue workers - Checkpointed long-running jobs - Batch processing with progress tracking ### 3. **Caching with SpaceTime** - Hot/cold cache tiers - Automatic memory pressure handling - Cache warming strategies ### 4. **Real-World Use Cases** - User activity log processing - Sales report generation - Product catalog management - Real-time analytics ## Installation 1. **Install dependencies:** ```bash composer install ``` 2. **Configure environment:** ```bash cp .env.example .env php artisan key:generate ``` 3. **Configure SpaceTime in `.env`:** ``` SPACETIME_MEMORY_LIMIT=256M SPACETIME_EXTERNAL_STORAGE=/tmp/spacetime SPACETIME_CHUNK_STRATEGY=sqrt_n SPACETIME_ENABLE_CHECKPOINTING=true ``` 4. **Run migrations:** ```bash php artisan migrate php artisan db:seed ``` ## Project Structure ``` laravel-app/ ├── app/ │ ├── Http/ │ │ ├── Controllers/ │ │ │ ├── ProductController.php # Streaming APIs │ │ │ ├── AnalyticsController.php # Real-time analytics │ │ │ └── ReportController.php # Large report generation │ │ └── Middleware/ │ │ └── SpaceTimeMiddleware.php # Memory monitoring │ ├── Jobs/ │ │ ├── ProcessLargeDataset.php # Checkpointed job │ │ ├── GenerateReport.php # Batch processing job │ │ └── ImportProducts.php # CSV import job │ ├── Services/ │ │ ├── ProductService.php # Business logic │ │ ├── AnalyticsService.php # Analytics processing │ │ └── SpaceTimeCache.php # Cache wrapper │ └── Providers/ │ └── SpaceTimeServiceProvider.php # Service registration ├── config/ │ └── spacetime.php # Configuration ├── routes/ │ ├── api.php # API routes │ └── web.php # Web routes └── tests/ └── Feature/ └── SpaceTimeTest.php # Integration tests ``` ## Usage Examples ### 1. Streaming Large Datasets ```php // ProductController.php public function stream() { return response()->stream(function () { $products = SpaceTimeStream::fromQuery( Product::query()->orderBy('id') ); echo "["; $first = true; foreach ($products->chunk(100) as $chunk) { foreach ($chunk as $product) { if (!$first) echo ","; echo $product->toJson(); $first = false; } // Flush output buffer ob_flush(); flush(); } echo "]"; }, 200, [ 'Content-Type' => 'application/json', 'X-Accel-Buffering' => 'no' ]); } ``` ### 2. Memory-Efficient CSV Export ```php // ReportController.php public function exportCsv() { $filename = 'products_' . date('Y-m-d') . '.csv'; return response()->streamDownload(function () { $exporter = new CsvExporter('php://output'); $exporter->writeHeaders(['ID', 'Name', 'Price', 'Stock']); Product::query() ->orderBy('id') ->chunkById(1000, function ($products) use ($exporter) { foreach ($products as $product) { $exporter->writeRow([ $product->id, $product->name, $product->price, $product->stock ]); } }); }, $filename, [ 'Content-Type' => 'text/csv', ]); } ``` ### 3. Checkpointed Background Job ```php // ProcessLargeDataset.php class ProcessLargeDataset implements ShouldQueue { use Dispatchable, InteractsWithQueue, Queueable, SerializesModels; use SpaceTimeCheckpointable; public function handle() { $checkpoint = $this->getCheckpoint(); $lastId = $checkpoint['last_id'] ?? 0; Order::where('id', '>', $lastId) ->orderBy('id') ->chunkById(100, function ($orders) { foreach ($orders as $order) { // Process order $this->processOrder($order); // Save checkpoint every 100 orders if ($order->id % 100 === 0) { $this->saveCheckpoint([ 'last_id' => $order->id, 'processed' => $this->processed, ]); } } }); } } ``` ### 4. Real-Time Analytics ```php // AnalyticsController.php public function realtime() { return response()->stream(function () { $monitor = new MemoryPressureMonitor('100M'); while (true) { $stats = $this->analyticsService->getCurrentStats(); // Send as Server-Sent Event echo "data: " . json_encode($stats) . "\n\n"; ob_flush(); flush(); // Check memory pressure if ($monitor->check() !== MemoryPressureLevel::NONE) { $this->analyticsService->compact(); } sleep(1); } }, 200, [ 'Content-Type' => 'text/event-stream', 'Cache-Control' => 'no-cache', 'X-Accel-Buffering' => 'no' ]); } ``` ### 5. Memory-Aware Caching ```php // SpaceTimeCache.php class SpaceTimeCache { private SpaceTimeDict $hot; private CacheInterface $cold; private MemoryPressureMonitor $monitor; public function get($key) { // Check hot cache first if (isset($this->hot[$key])) { return $this->hot[$key]; } // Check cold storage $value = $this->cold->get($key); if ($value !== null) { // Promote to hot cache if memory allows if ($this->monitor->canAllocate(strlen($value))) { $this->hot[$key] = $value; } } return $value; } } ``` ## API Endpoints ### Products API - `GET /api/products` - Paginated list - `GET /api/products/stream` - Stream all products as NDJSON - `GET /api/products/export/csv` - Export as CSV - `POST /api/products/bulk-update` - Bulk update with checkpointing - `POST /api/products/import` - Import CSV with progress ### Analytics API - `GET /api/analytics/summary` - Get summary statistics - `GET /api/analytics/realtime` - Real-time SSE stream - `POST /api/analytics/report` - Generate large report - `GET /api/analytics/top-products` - Top products with external sorting ### Reports API - `POST /api/reports/generate` - Generate report (queued) - `GET /api/reports/{id}/status` - Check generation status - `GET /api/reports/{id}/download` - Download completed report ## Testing Run the test suite: ```bash php artisan test ``` Example test: ```php public function test_can_stream_large_dataset() { // Seed test data Product::factory()->count(10000)->create(); // Make streaming request $response = $this->getJson('/api/products/stream'); $response->assertStatus(200); $response->assertHeader('Content-Type', 'application/json'); // Verify memory usage stayed low $this->assertLessThan(50 * 1024 * 1024, memory_get_peak_usage()); } ``` ## Performance Tips 1. **Configure memory limits** based on your server capacity 2. **Use streaming responses** for large datasets 3. **Enable checkpointing** for long-running jobs 4. **Monitor memory pressure** in production 5. **Use external storage** on fast SSDs 6. **Configure queue workers** with appropriate memory limits ## Deployment ### Nginx Configuration ```nginx location /api/products/stream { proxy_pass http://backend; proxy_buffering off; proxy_read_timeout 3600; } location /api/analytics/realtime { proxy_pass http://backend; proxy_buffering off; proxy_read_timeout 0; proxy_http_version 1.1; } ``` ### Supervisor Configuration ```ini [program:spacetime-worker] command=php /path/to/artisan queue:work --memory=256 numprocs=4 autostart=true autorestart=true ``` ## Monitoring Add to your monitoring: ```php // app/Console/Commands/MonitorSpaceTime.php $stats = [ 'memory_usage' => memory_get_usage(true), 'peak_memory' => memory_get_peak_usage(true), 'external_files' => count(glob(config('spacetime.external_storage') . '/*')), 'cache_size' => $this->cache->size(), ]; Log::channel('metrics')->info('spacetime.stats', $stats); ``` ## Troubleshooting ### High Memory Usage - Check `SPACETIME_MEMORY_LIMIT` setting - Enable more aggressive spillover - Use smaller chunk sizes ### Slow Performance - Ensure external storage is on SSD - Increase memory limit if possible - Use compression for large values ### Failed Checkpoints - Check storage permissions - Ensure sufficient disk space - Verify checkpoint directory exists ## Learn More - [SqrtSpace SpaceTime Documentation](https://github.com/MarketAlly/Ubiquity) - [Laravel Documentation](https://laravel.com/docs) - [Memory-Efficient PHP Patterns](https://example.com/patterns)