SFCC AI & Vertex AI Product Recommendations: Integration Architecture

By Kenneth D'Silva

At 2:14 AM on the Tuesday before Thanksgiving, our primary production database cluster threw a fit. We were running a massive pre-sale on conical fermenters, specifically the 14-gallon chronicals that home brewers covet. Traffic spiked to 4,500 concurrent sessions, which was high but manageable for our Salesforce Commerce Cloud (SFCC) instance. The problem wasn't the storefront tier. The problem was our reliance on real-time, un-cached calls to a legacy recommendation engine that choked when asked to compute collaborative filtering logic across our entire historical order database of yeast, malt, and equipment combinations. The database locked up. Thread pools exhausted. Connection timeouts cascaded through the application servers. We lost thirty-five minutes of peak pre-holiday sales.

I sat there looking at Datadog dashboards painted solid red, watching the active connection count drop to zero. The immediate fix was obvious: disable the recommendation zones in the ISML templates, redeploy, and let the site run bare. We stabilized. But the business lost the upsell pipeline for the rest of the week. We needed a decoupled, highly available, and smarter recommendation architecture. We needed something that didn't drag down the critical path of the checkout flow.

This led to a four-month engineering sprint to rip out our synchronous, monolithic recommendation setup and replace it with a hybrid approach using Google Vertex AI (specifically Google Cloud Retail API) alongside SFCC Einstein. The goal was twofold. First, offload the heavy lifting of model training and real-time inference to Google's infrastructure. Second, implement a strict client-side hydration model so that the storefront servers never block on a recommendation API call. Here is exactly how we built it, the mistakes we made, and the code that runs it in production.

The Architecture: Decoupling SFCC from Inference

The core philosophy of this integration is simple: SFCC is the system of record for catalog and pricing data. Vertex AI is the brain. The storefront client is the execution environment. Never the twain shall meet on a server thread.

We designed a three-pillar architecture:

Data Ingestion Pipelines: Batch and streaming processes to move catalog updates, inventory changes, and historical user events from SFCC to Google Cloud.
Context-Aware Routing API: A lightweight Node.js/Express service hosted on Google Kubernetes Engine (GKE) behind Cloud CDN. This API acts as a gateway, deciding whether to serve recommendations from Vertex AI, fallback to a cached Einstein response, or serve default merchandising rules.
Client-Side Hydration: Vanilla JavaScript components that inject recommendation carousels into the DOM after the main page load completes, firing interaction events back to the routing API asynchronously.

Pillar 1: Data Ingestion Pipelines

Vertex AI is entirely dependent on the quality and freshness of the catalog and user event data you feed it. We are dealing with home brewing supplies. The relationships between products are highly specific. A customer buying a specific strain of liquid ale yeast (like WLP001) is highly likely to need a yeast starter kit, dry malt extract, and perhaps a stir plate. They are not likely to need a wine degassing wand. The model needs to understand these semantic relationships, which means it needs detailed catalog attributes.

The Catalog Export Job

We built a custom SFCC Job step using the Job Framework to export the catalog. We opted against the standard XML export because parsing massive XML files in our downstream Google Cloud Functions was memory-intensive. Instead, we generate a JSONL (JSON Lines) file directly from SFCC.

Here is the core logic of the Steptype script that iterates over the catalog and streams it to a custom JSONL file.


/**
 * ExportCatalogToJSONL.js
 * 
 * SFCC Job step to export product catalog in Google Cloud Retail API format.
 */
var ProductMgr = require('dw/catalog/ProductMgr');
var File = require('dw/io/File');
var FileWriter = require('dw/io/FileWriter');
var Logger = require('dw/system/Logger');
var Status = require('dw/system/Status');

function execute(parameters) {
    var exportDir = new File(File.IMPEX + '/src/export/vertex');
    if (!exportDir.exists()) {
        exportDir.mkdirs();
    }

    var timestamp = new Date().getTime();
    var file = new File(exportDir, 'catalog_export_' + timestamp + '.jsonl');
    var fileWriter = new FileWriter(file, 'UTF-8');
    
    var products = ProductMgr.queryAllSiteProducts();
    var count = 0;

    try {
        while (products.hasNext()) {
            var product = products.next();
            
            // Skip offline products or variants (we only want masters and standard products for this specific model)
            if (!product.isOnline() || product.isVariant()) {
                continue;
            }

            var retailProduct = mapToRetailAPI(product);
            fileWriter.writeLine(JSON.stringify(retailProduct));
            count++;

            if (count % 1000 === 0) {
                Logger.info('Exported {0} products', count);
            }
        }
    } catch (e) {
        Logger.error('Error exporting catalog: {0}', e.message);
        return new Status(Status.ERROR, 'EXPORT_FAILED');
    } finally {
        products.close();
        fileWriter.close();
    }

    Logger.info('Successfully exported {0} products to {1}', count, file.fullPath);
    return new Status(Status.OK);
}

function mapToRetailAPI(product) {
    var categories = [];
    var categoryAssignments = product.getCategoryAssignments();
    for (var i = 0; i < categoryAssignments.length; i++) {
        var cat = categoryAssignments[i].getCategory();
        var path = [];
        while (cat != null && cat.ID !== 'root') {
            path.unshift(cat.displayName);
            cat = cat.parent;
        }
        categories.push(path.join(' > '));
    }

    // Extracting custom attributes specific to brewing
    var custom = product.custom;
    var attributes = {};
    if (custom.beerStyle) attributes.beerStyle = { text: [custom.beerStyle] };
    if (custom.yeastType) attributes.yeastType = { text: [custom.yeastType] };
    if (custom.alphaAcid) attributes.alphaAcid = { numbers: [custom.alphaAcid] };

    return {
        id: product.ID,
        title: product.name,
        description: product.shortDescription ? product.shortDescription.markup : '',
        categories: categories,
        priceInfo: {
            currencyCode: 'USD',
            price: product.priceModel.price.value,
            originalPrice: product.priceModel.basePrice.value
        },
        availability: product.availabilityModel.isOrderable() ? 'IN_STOCK' : 'OUT_OF_STOCK',
        attributes: attributes,
        uri: require('dw/web/URLUtils').abs('Product-Show', 'pid', product.ID).toString()
    };
}

module.exports.execute = execute;

This script runs nightly at 3:00 AM. A subsequent job step uploads the generated JSONL file to a Google Cloud Storage (GCS) bucket using a custom SFCC Service configured with a Google Service Account JWT for authentication. We do not use WebDAV for external system transfers. WebDAV is a relic. We use direct HTTPS PUT requests to the GCS JSON API.

Cloud Composer and DAG Orchestration

Once the file lands in GCS, an Eventarc trigger fires a Cloud Run function that alerts our Cloud Composer (Apache Airflow) environment. We use Airflow to orchestrate the actual import into Vertex AI. We chose Airflow because we need strict dependency management. We cannot train the recommendation models until the catalog import completes successfully, and we cannot import the catalog if the file fails basic validation checks.

Here is the Airflow DAG that handles the ingestion and model retraining:


from airflow import models
from airflow.providers.google.cloud.operators.gcs import GCSListObjectsOperator
from airflow.providers.google.cloud.operators.vertex_ai.dataset import ImportDataVertexAIOperator
from airflow.operators.python import PythonOperator
from airflow.utils.dates import days_ago
from google.cloud import retail_v2
import logging

PROJECT_ID = 'modracx-production-39281'
LOCATION = 'global'
CATALOG_ID = 'default_catalog'
BRANCH_ID = 'default_branch'
GCS_BUCKET = 'modracx-retail-data-inbound'

default_args = {
    'start_date': days_ago(1),
    'retries': 2,
}

def import_catalog_to_retail_api(ti):
    client = retail_v2.ProductServiceClient()
    files = ti.xcom_pull(task_ids='list_gcs_files')
    
    if not files:
        logging.info("No files found to import.")
        return

    # Sort to get the most recent file based on our timestamp naming convention
    latest_file = sorted(files)[-1]
    gcs_uri = f"gs://{GCS_BUCKET}/{latest_file}"
    logging.info(f"Importing from {gcs_uri}")

    gcs_source = retail_v2.GcsSource()
    gcs_source.input_uris = [gcs_uri]
    gcs_source.data_schema = "product"

    input_config = retail_v2.ProductInputConfig()
    input_config.gcs_source = gcs_source

    request = retail_v2.ImportProductsRequest(
        parent=f"projects/{PROJECT_ID}/locations/{LOCATION}/catalogs/{CATALOG_ID}/branches/{BRANCH_ID}",
        input_config=input_config,
    )

    operation = client.import_products(request=request)
    logging.info("Waiting for operation to complete...")
    response = operation.result()
    logging.info(f"Import completed. Errors: {response.error_samples}")

with models.DAG(
    'vertex_retail_catalog_ingestion',
    default_args=default_args,
    schedule_interval=None,
    catchup=False,
) as dag:

    list_files = GCSListObjectsOperator(
        task_id='list_gcs_files',
        bucket=GCS_BUCKET,
        prefix='catalog_export_',
    )

    import_catalog = PythonOperator(
        task_id='import_catalog',
        python_callable=import_catalog_to_retail_api,
    )

    # In a real setup, we would trigger model retraining here
    # trigger_model_retrain = ...

    list_files >> import_catalog

This DAG ensures that the Retail API is always aware of the latest products, categories, and custom brewing attributes. If the import fails, PagerDuty fires, and someone looks at the logs. We do not tolerate stale catalogs.

Pillar 2: The Context-Aware Routing API

We cannot have the browser communicating directly with Google Cloud Retail API. That exposes API keys and tightly couples the frontend to a specific backend structure. We needed a middleware layer. We built a Node.js API hosted on GKE.

This API serves three critical functions:

Security and Authentication: It holds the Google Cloud service account credentials.
A/B Testing and Traffic Splitting: We use LaunchDarkly to evaluate flags at the edge. The API decides if a specific user session should receive recommendations from Vertex AI, from a legacy cached Einstein payload, or a static fallback.
Response Normalization: It takes the highly verbose Vertex AI response and flattens it into a lean JSON array that the client can easily iterate over.

Here is a snippet of the Express controller that handles the recommendation request. Notice how we handle timeouts. We enforce a strict 300ms SLA on this API. If Google doesn't respond in time, we return an empty array. The client will just hide the carousel. A missing carousel is infinitely better than a broken page or a hanging request.


const express = require('express');
const { PredictionServiceClient } = require('@google-cloud/retail').v2;
const LaunchDarkly = require('launchdarkly-node-server-sdk');

const router = express.Router();
const predictionClient = new PredictionServiceClient();
const ldClient = LaunchDarkly.init(process.env.LD_SDK_KEY);

const PROJECT_ID = process.env.GOOGLE_PROJECT_ID;
const LOCATION = 'global';
const CATALOG_ID = 'default_catalog';

router.post('/api/recommendations/:placementId', async (req, res) => {
    const { placementId } = req.params;
    const { visitorId, productId, cartItemIds } = req.body;
    
    // Evaluate feature flag for traffic splitting
    const user = { key: visitorId };
    const useVertex = await ldClient.variation('use-vertex-recommendations', user, false);

    if (!useVertex) {
        // Fallback to old system or return empty
        return res.json({ provider: 'fallback', products: [] });
    }

    const placementPath = `projects/${PROJECT_ID}/locations/${LOCATION}/catalogs/${CATALOG_ID}/servingConfigs/${placementId}`;
    
    const userEvent = {
        eventType: 'detail-page-view',
        visitorId: visitorId,
        productDetails: productId ? [{ product: { id: productId } }] : []
    };

    const request = {
        placement: placementPath,
        userEvent: userEvent,
        pageSize: 10,
    };

    try {
        // Enforce strict timeout
        const timeoutPromise = new Promise((_, reject) => 
            setTimeout(() => reject(new Error('Vertex API Timeout')), 300)
        );
        
        const predictionPromise = predictionClient.predict(request);
        const [response] = await Promise.race([predictionPromise, timeoutPromise]);

        // Normalize response for the client
        const normalizedProducts = response.results.map(result => ({
            id: result.id,
            title: result.product.title,
            price: result.product.priceInfo.price,
            url: result.product.uri,
            imageUrl: result.product.images[0]?.uri || '/images/default.jpg'
        }));

        res.json({ provider: 'vertex', products: normalizedProducts });

    } catch (error) {
        console.error(`Recommendation fetch failed: ${error.message}`);
        // Fail gracefully
        res.status(200).json({ provider: 'fallback_error', products: [] });
    }
});

module.exports = router;

This middleware is deployed across three GCP regions (us-central1, us-east4, us-west1) behind an external HTTP(S) load balancer. We use Cloud CDN to cache static assets, but the `/api/recommendations` endpoint is explicitly set to `Cache-Control: no-store`. Recommendations must be real-time and context-aware based on the user's browsing history.

Pillar 3: Client-Side Hydration

The final piece of the puzzle is how the storefront renders these recommendations. We completely removed all `<isslot>` tags from our ISML templates that were responsible for rendering product carousels based on server-side logic.

Instead, we render empty container divs with data attributes describing the context.


<!-- product-detail-page.isml -->
<div class="mod-recommendation-zone" 
     data-placement="pdp_frequently_bought_together"
     data-context-product="${pdict.Product.ID}">
    <!-- Skeleton loader injected via CSS -->
    <div class="skeleton-carousel"></div>
</div>

A vanilla JavaScript controller initializes on `DOMContentLoaded`. It finds all elements with the `mod-recommendation-zone` class, extracts the configuration, and fires requests to our routing API using the `IntersectionObserver` API. We only fetch recommendations when the carousel is about to scroll into the viewport. This saves massive amounts of API quota and bandwidth.


// recommendations.js
class RecommendationManager {
    constructor() {
        this.zones = document.querySelectorAll('.mod-recommendation-zone');
        this.visitorId = this.getVisitorId();
        this.initObserver();
    }

    getVisitorId() {
        let vid = localStorage.getItem('mod_visitor_id');
        if (!vid) {
            vid = crypto.randomUUID();
            localStorage.setItem('mod_visitor_id', vid);
        }
        return vid;
    }

    initObserver() {
        const options = {
            root: null,
            rootMargin: '200px', // Fetch slightly before it enters viewport
            threshold: 0.1
        };

        const observer = new IntersectionObserver((entries, obs) => {
            entries.forEach(entry => {
                if (entry.isIntersecting) {
                    this.fetchAndRender(entry.target);
                    obs.unobserve(entry.target);
                }
            });
        }, options);

        this.zones.forEach(zone => observer.observe(zone));
    }

    async fetchAndRender(zoneElement) {
        const placement = zoneElement.dataset.placement;
        const productId = zoneElement.dataset.contextProduct;
        
        try {
            const response = await fetch(`/api/recommendations/${placement}`, {
                method: 'POST',
                headers: { 'Content-Type': 'application/json' },
                body: JSON.stringify({
                    visitorId: this.visitorId,
                    productId: productId
                })
            });

            if (!response.ok) throw new Error('Network response was not ok');
            
            const data = await response.json();
            this.renderCarousel(zoneElement, data.products);

        } catch (error) {
            console.error('Failed to load recommendations', error);
            zoneElement.style.display = 'none'; // Collapse the space if fetch fails
        }
    }

    renderCarousel(container, products) {
        if (!products || products.length === 0) {
            container.style.display = 'none';
            return;
        }

        const html = products.map(product => `
            <div class="product-card">
                <a href="${product.url}">
                    <img src="${product.imageUrl}" alt="${product.title}" loading="lazy">
                    <h4>${product.title}</h4>
                    <span class="price">$${product.price.toFixed(2)}</span>
                </a>
            </div>
        `).join('');

        container.innerHTML = `<div class="carousel-track">${html}</div>`;
    }
}

document.addEventListener('DOMContentLoaded', () => {
    new RecommendationManager();
});

This approach completely isolates the storefront performance from the recommendation engine performance. If the Node.js API goes down, or if Vertex AI experiences an outage, the customer sees a product detail page without a "Frequently Bought Together" section. The core funnel remains intact. They can still add to cart. They can still checkout. We preserve revenue during partial degradation.

Data Synchronization and Ingestion Challenges

Let's talk about the hard parts. Moving data is never as simple as a cron job and a REST call. When you operate a catalog of roughly 12,000 SKUs, with constant price fluctuations based on grain commodity markets and frequent inventory stock-outs, you cannot rely solely on a nightly batch process. A customer trying to buy a specific hop variety needs to know if the recommended substitute is actually in stock.

Streaming Inventory Updates

The nightly catalog export handles the bulk data: names, descriptions, categories, and custom attributes. But inventory and pricing require near real-time synchronization. To solve this, we implemented a streaming pipeline using Google Cloud Pub/Sub.

Within SFCC, we have hooks tied to inventory updates. Whenever an inventory record is modified (usually via an integration with our ERP, NetSuite), SFCC fires a lightweight webhook to our middleware layer.


// SFCC Hook: app.inventory.update
var HTTPClient = require('dw/net/HTTPClient');
var Logger = require('dw/system/Logger');

exports.afterPATCH = function(inventoryRecord) {
    var payload = {
        productId: inventoryRecord.productID,
        available: inventoryRecord.allocation > 0,
        timestamp: new Date().toISOString()
    };

    var client = new HTTPClient();
    client.setTimeout(1000); // 1 second timeout, do not block the thread
    
    try {
        client.open('POST', 'https://api.modracx.com/webhooks/inventory');
        client.setRequestHeader('Content-Type', 'application/json');
        client.setRequestHeader('Authorization', 'Bearer ' + getWebhookToken());
        client.send(JSON.stringify(payload));
    } catch (e) {
        Logger.error('Failed to stream inventory update for {0}: {1}', payload.productId, e.message);
    }
};

Our Node.js middleware receives this webhook, authenticates it, and immediately publishes a message to a Pub/Sub topic.


// Node.js Webhook Receiver
const { PubSub } = require('@google-cloud/pubsub');
const pubsub = new PubSub();
const topic = pubsub.topic('retail-inventory-updates');

router.post('/webhooks/inventory', authenticateWebhook, async (req, res) => {
    const { productId, available, timestamp } = req.body;
    
    const messageData = JSON.stringify({
        id: productId,
        availability: available ? 'IN_STOCK' : 'OUT_OF_STOCK'
    });

    try {
        await topic.publishMessage({ data: Buffer.from(messageData) });
        res.status(202).send('Accepted');
    } catch (error) {
        console.error('PubSub publish failed', error);
        res.status(500).send('Internal Error');
    }
});

Finally, a Google Cloud Function subscribes to this Pub/Sub topic and uses the Retail API's `patch` method to update only the availability field of the specific product. This ensures that Vertex AI stops recommending out-of-stock items within seconds of the inventory changing in SFCC, all without needing to re-import the entire catalog.

Model Training and Optimization

Vertex AI is powerful, but it is not magic. You cannot throw arbitrary data at it and expect high conversion rates. We spent weeks tuning the models based on the specific nuances of home brewing.

We primarily use three types of models in the Retail API:

Others You May Like (Collaborative Filtering): Used on the Product Detail Page. This looks at co-view and co-purchase behavior.
Frequently Bought Together: Used in the cart modal. This strictly looks at items purchased in the same transaction.
Recently Viewed: A simple determinist model used on the homepage to reduce friction for returning users.

The biggest hurdle we faced was catalog sparsity. Many of our niche hardware components (like specific stainless steel tri-clamps or specialized gaskets) have very low purchase volume. The collaborative filtering model struggled to find statistical significance for these items. It kept recommending highly popular items (like generic cleaning powder, PBW) on every single page, drowning out the highly relevant, niche items.

We solved this by using the `filter` syntax in our prediction requests to enforce categorical alignment, and by heavily weighting the custom attributes we exported from SFCC.

For example, if you are looking at a product with the `beerStyle` attribute set to "IPA", we enforce a soft filter to boost recommendations that share the same `beerStyle` or belong to the "Hops" category. Here is what the modified prediction request looks like in our Node.js middleware:


// Modified Prediction Request with Filtering
const request = {
    placement: placementPath,
    userEvent: userEvent,
    pageSize: 10,
    filter: `filterOutOfStockItems AND categories: ANY("Hops", "Yeast")`, // Example rigid filter
    // Or we use dynamic boosting based on the context product
};

We rely on A/B testing via LaunchDarkly to validate these filter configurations. We run experiments continuously, measuring not just click-through rate on the carousels, but actual add-to-cart rate and final revenue per session.

The Results: Why We Built This

The numbers justify the engineering effort. Since migrating from the synchronous legacy engine to the Vertex AI / Node.js hybrid architecture, we have seen massive improvements across both technical and business metrics.

SFCC Application Server Load: CPU utilization during peak traffic dropped by 42%. By removing the synchronous server-side API calls to the old recommendation engine, we freed up massive amounts of thread pool capacity. Our storefront can handle significantly higher concurrency without scaling up infrastructure.
Page Load Time (DOM Interactive): Improved by 600ms on average. The initial HTML payload is smaller because it doesn't contain server-rendered product markup for the carousels. The client fetches and renders them asynchronously.
Recommendation Conversion Rate: Increased by 18%. Vertex AI's collaborative filtering, once properly tuned with our custom brewing attributes, simply surface better products. Customers buying malt are seeing the exact specialized grain bags they need, not generic sanitizers.
System Resilience: We have achieved zero downtime related to recommendation engine failures. The client-side degradation strategy works perfectly. When Google Cloud US-Central had a brief networking blip last month, our middleware timed out, the carousels collapsed, and the core site continued processing transactions flawlessly.

Building this architecture requires a shift in mindset. You have to stop viewing the ecommerce platform as a monolithic rendering engine and start viewing it as a pure data source. By decoupling the heavy computation (machine learning) and the orchestration (our middleware) from the presentation layer (the browser), we built a system that scales linearly and degrades gracefully. This is how modern ecommerce architecture should look.

7. Real-Time Data Ingestion Pipelines

To feed Vertex AI accurately, we needed a robust data pipeline capable of capturing user interactions (views, add-to-carts, purchases) in real-time. We bypassed SFCC's native tracking and built a custom client-side event bus using Google Tag Manager (GTM) and Google Cloud Pub/Sub.

When a user views a home brewing kit, a payload is fired directly to a Pub/Sub HTTP endpoint. This minimizes the load on the SFCC application servers and ensures data reaches Google Cloud instantly.


// Client-side tracking script
function trackProductView(product) {
    const payload = {
        event: 'product_view',
        timestamp: new Date().toISOString(),
        client_id: getCookie('_ga'),
        user_id: getCookie('sfcc_customer_id') || null,
        product: {
            id: product.id,
            name: product.name,
            category: product.category,
            price: product.price
        }
    };

    fetch('https://pubsub.googleapis.com/v1/projects/my-gcp-project/topics/sfcc-events:publish', {
        method: 'POST',
        headers: {
            'Content-Type': 'application/json',
            'Authorization': `Bearer ${getOauthToken()}` // Handled via secure edge worker
        },
        body: JSON.stringify({
            messages: [{ data: btoa(JSON.stringify(payload)) }]
        })
    }).catch(console.error);
}

8. BigQuery SQL: Preparing Data for Vertex AI

Vertex AI Recommendations requires specific schemas for its training data: Users, Items, and User-Events. We utilized BigQuery scheduled queries to transform the raw Pub/Sub event stream into the precise formats required by Vertex AI.

Here is the critical SQL query we run daily to aggregate the user-event data, handling sessionization and deduplication for our brewing supplies catalog:


-- BigQuery Transformation for Vertex AI User-Events
WITH DedupedEvents AS (
    SELECT 
        user_id,
        client_id,
        event_type,
        JSON_EXTRACT_SCALAR(product, '$.id') AS product_id,
        timestamp,
        ROW_NUMBER() OVER(PARTITION BY client_id, JSON_EXTRACT_SCALAR(product, '$.id'), event_type ORDER BY timestamp DESC) as rn
    FROM `my-gcp-project.raw_events.product_interactions`
    WHERE timestamp >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 90 DAY)
)
SELECT 
    event_type,
    COALESCE(user_id, client_id) AS visitor_id,
    FORMAT_TIMESTAMP('%Y-%m-%dT%H:%M:%SZ', timestamp) AS eventTime,
    product_id AS productId
FROM DedupedEvents
WHERE rn = 1
  AND product_id IS NOT NULL;

9. Context-Aware Routing and Caching Strategy

Hitting the Vertex AI API for every single page load on a high-traffic SFCC site will instantly obliterate your API budget and introduce severe latency. We implemented a multi-tiered caching strategy.

For anonymous users, we serve pre-computed "trending" or "bestseller" recommendations cached at the CDN edge (Fastly). We only invoke the Vertex AI endpoint for authenticated users with a known history, and even then, we cache the response in Redis for 15 minutes.


// SFCC Controller: Product.js (Simplified)
var server = require('server');
var cache = require('*/cartridge/scripts/helpers/cacheHelper');
var vertexAPI = require('*/cartridge/scripts/api/vertexAI');

server.get('Recommendations', function (req, res, next) {
    var productId = req.querystring.pid;
    var customerId = req.currentCustomer.profile ? req.currentCustomer.profile.customerNo : null;
    var cacheKey = 'recs_' + (customerId || 'anon') + '_' + productId;

    var cachedRecs = cache.get(cacheKey);
    if (cachedRecs) {
        res.json(cachedRecs);
        return next();
    }

    var recommendations = [];
    if (customerId) {
        // Personalized Vertex AI call
        recommendations = vertexAPI.getRecommendations(customerId, productId);
    } else {
        // Fallback to SFCC Einstein or static trending list
        recommendations = require('*/cartridge/scripts/helpers/einstein').getTrending(productId);
    }

    cache.set(cacheKey, recommendations, 900); // 15 minute TTL
    res.json(recommendations);
    next();
});

10. Comprehensive Integration FAQ

Q: How do we handle Vertex AI cold starts for new products?: New brewing kits lack interaction history. We configured Vertex AI to use content-based filtering as a fallback, analyzing product descriptions, categories (e.g., 'Extract Kits' vs 'All-Grain'), and pricing to map new items to existing clusters.
Q: What is the cost difference between Einstein and Vertex AI?: Einstein is included in the SFCC GMV tier. Vertex AI bills per node hour for training and per 1000 predictions. For a site doing 5M monthly visits, Vertex AI costs approximately $2,500/month, but the 12% AOV uplift justified the cost instantly.
Q: How do we sync catalog updates to Vertex AI?: We do not rely on batch XML uploads. We built a webhook in SFCC that triggers a Cloud Function whenever a product is updated in the Business Manager, pushing the mutation directly to the Vertex AI Retail API.
Q: How do we A/B test Vertex vs Einstein?: We use a server-side split in SFCC based on the user's session ID (even/odd). We pipe the exposure events into BigQuery and calculate statistical significance on conversion rate and revenue per visitor using custom looker studio dashboards.
Q: How does this impact page load speeds?: We decoupled recommendations from the main SSR HTML. The page loads instantly, and a lightweight client-side script fetches the recommendations asynchronously. This guarantees the LCP metric is never impacted by recommendation latency.
Q: Can Vertex AI handle regional pricing variations?: Yes, but you must pass the localized price in the event payload. We maintain separate Vertex AI serving configs for our US and EU storefronts to prevent cross-contamination of pricing data.
Q: How do we prevent recommending out-of-stock items?: Our SFCC webhook immediately updates the `availability` status in Vertex AI when inventory hits zero. Additionally, the SFCC controller validates stock levels before rendering the final HTML block.
Q: How do you handle bot traffic skewing the model?: Bot traffic is lethal to recommendation models. We filter out known bot user-agents at the CDN edge, and our BigQuery pipeline excludes any session that generates more than 50 events in a minute.
Q: How long does the initial Vertex AI model training take?: For 5 years of historical data (approx 200M events), the initial tuning took 72 hours. Subsequent daily retrainings take about 4 hours.
Q: What if the Vertex API goes down?: Our SFCC controller wraps the API call in a 200ms timeout. If Vertex fails, we silently degrade to a hardcoded 'Top Sellers' slot configured in the SFCC Business Manager. The user never sees an error.

11. Implementing Client-Side Hydration Islands

Once the recommendations are fetched from the Vertex AI endpoint via the SFCC controller, we use a concept known as Hydration Islands to render the UI without blocking the main thread. We use Preact for this specific component to keep the payload under 4KB.


import { h, render } from 'preact';
import { useState, useEffect } from 'preact/hooks';

const RecommendationCarousel = ({ productId }) => {
    const [products, setProducts] = useState([]);
    
    useEffect(() => {
        fetch(`/on/demandware.store/Sites-Brewing-Site/default/Recommendations?pid=${productId}`)
            .then(res => res.json())
            .then(data => setProducts(data));
    }, [productId]);
    
    if (products.length === 0) return null;
    
    return (
        
            {products.map(p => (
                
                    
                    {p.name}
                    ${p.price}
                
            ))}
        
    );
};

// Mount only when visible
const observer = new IntersectionObserver((entries) => {
    entries.forEach(entry => {
        if (entry.isIntersecting) {
            const el = entry.target;
            render(, el);
            observer.unobserve(el);
        }
    });
});
document.querySelectorAll('.recommendation-slot').forEach(el => observer.observe(el));

12. Unpacking the Telemetry Payload: Raw JSON Schemas

The success of the Vertex AI model is entirely dependent on the structural integrity of the telemetry data you feed it. Sending unstructured, generic payloads will result in a model that recommends randomly. We enforce a strict JSON Schema validation on the Pub/Sub edge workers before the data ever reaches BigQuery.

Here is the exact, unvarnished JSON schema we mandate for a add-to-cart event within the home brewing catalog. Notice the explicit inclusion of the visitor_id which merges the anonymous client_id with the authenticated user_id upon login, ensuring the session stitching does not break when a user transitions from a guest to an authenticated state.


{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "Vertex_AI_UserEvent",
  "type": "object",
  "properties": {
    "eventType": {
      "type": "string",
      "enum": ["detail-page-view", "add-to-cart", "purchase-complete", "search"]
    },
    "visitorId": {
      "type": "string",
      "description": "The stitched identifier (hashed cookie or user ID)"
    },
    "eventTime": {
      "type": "string",
      "format": "date-time"
    },
    "productDetails": {
      "type": "object",
      "properties": {
        "product": {
          "type": "object",
          "properties": {
            "id": { "type": "string" },
            "title": { "type": "string" },
            "categories": {
              "type": "array",
              "items": { "type": "string" }
            },
            "priceInfo": {
              "type": "object",
              "properties": {
                "currencyCode": { "type": "string" },
                "price": { "type": "number" }
              }
            }
          },
          "required": ["id", "title", "categories"]
        },
        "quantity": {
          "type": "integer",
          "minimum": 1
        }
      },
      "required": ["product"]
    }
  },
  "required": ["eventType", "visitorId", "eventTime", "productDetails"]
}

13. Advanced BigQuery Tuning: Handling High-Volume Days

During promotions like "National Homebrew Day", our event volume spikes by 600%. If our BigQuery scheduled queries execute with brute-force `SELECT *` operations, our Google Cloud bill would eclipse the revenue generated by the recommendations.

To optimize this, we utilize BigQuery table partitioning and clustering. We partition the raw event tables by the `timestamp` column (specifically `DAY`) and cluster them by `visitor_id` and `event_type`. This allows our deduplication query to scan only a fraction of the data.


-- DDL for the optimized raw events table
CREATE TABLE `my-gcp-project.raw_events.product_interactions_optimized`
(
    event_type STRING,
    visitor_id STRING,
    product_id STRING,
    payload JSON,
    timestamp TIMESTAMP
)
PARTITION BY DATE(timestamp)
CLUSTER BY visitor_id, event_type;

By enforcing partition filters on the daily Vertex AI ingestion job (WHERE DATE(timestamp) = CURRENT_DATE()), we reduce the query bytes billed from 4 terabytes to roughly 12 gigabytes, keeping the operational cost of the recommendation engine essentially flat regardless of traffic spikes.

14. Additional FAQ: Expanding the Edge Cases

Q11: How do you handle bundle products (e.g., a complete brewing starter kit) in Vertex AI?: Bundle products are notoriously difficult for recommendation engines because they encompass multiple SKUs. In our data pipeline, we send the Parent Bundle ID as the primary product.id, but we include the component SKUs in a custom tags array within the Vertex JSON payload. This allows the model to map affinities between the bundle and its constituent parts, so if someone buys the Starter Kit, it knows to recommend the specific replacement hops or yeast included in that exact kit later.
Q12: What happens when the SFCC catalog hierarchy changes drastically?: If you restructure your catalog (e.g., moving "Malt Extract" from a top-level category to a sub-category of "Ingredients"), Vertex AI's context can temporarily fracture. We mitigate this by sending the full, breadcrumb-style category path (e.g., "Ingredients > Malt > Extract") as an array in the item catalog sync. If a major hierarchy shift occurs, we trigger a forced re-training of the Vertex model using the historical events mapped against the new catalog state, effectively backfilling the correct categories.
Q13: How does this integration respect GDPR and CCPA consent frameworks?: The client-side GTM trigger that fires the Pub/Sub event is strictly gated by our Consent Management Platform (OneTrust). If the user rejects the "Targeting/Advertising" cookie category, the `trackProductView` function is never invoked. Furthermore, for users who invoke their Right to be Forgotten, we execute a BigQuery `DELETE` statement targeting their specific `visitor_id` across all event tables, and purge their user profile from the Vertex AI endpoint via the UserEvents API.

15. Mitigating the Cold-Start Problem for Anonymous Traffic

One of the most complex architectural challenges with machine-learning recommendation engines is the cold-start problem. When an entirely new, anonymous user lands on your SFCC storefront, they possess no visitor_id history. They have viewed zero products and generated zero telemetry events. Firing a request to Vertex AI for a user with a blank behavioral profile is not only a waste of your API budget, but it will often return a highly generic, unoptimized response that degrades the initial user experience.

To combat this, we engineered a sophisticated session fallback behavior directly within the SFCC controller logic. When the Edge CDN detects a request lacking our proprietary tracking cookie, it entirely bypasses the Vertex AI API layer. Instead, it routes the request to a high-performance Redis cache containing pre-computed "Contextual Best Sellers." These fallback lists are generated nightly by a BigQuery job that analyzes aggregate purchase velocity over the last 72 hours, segmented by the category the anonymous user is currently viewing. If the user is on the homepage, they receive global trending items; if they are on a "Hops & Yeast" category page, they receive the highest-converting products strictly from that category.

The moment the anonymous user interacts with their first product (e.g., clicking on a specific brewing kit), the client-side GTM script instantly fires a detail-page-view event to Pub/Sub and assigns them a temporary session ID. On their very next page load, the SFCC controller detects this new session ID. However, because Vertex AI models rely on deep historical context, a single event is rarely enough to generate a highly personalized matrix. Therefore, we utilize Vertex AI's "Context-Based" serving config for the first 5 interactions. This model heavily weights the immediate session context (the product they just viewed) over long-term affinities, functioning essentially as a highly intelligent "Similar Products" engine.

We rigorously monitor the performance profiling metrics of this hybrid approach. By isolating anonymous traffic from the Vertex AI API, we reduced our Google Cloud inference costs by 34% while simultaneously improving the Time-to-First-Byte (TTFB) for new visitors by 120 milliseconds. Our custom Looker Studio dashboards specifically track the "Recommendation Engine Source" dimension (Redis Fallback vs. Vertex AI) against Conversion Rate. The data proved that for users with fewer than 3 interactions, the pre-computed category best-sellers actually outperformed the ML model by 1.8%, validating the architectural decision to delay Vertex AI invocation until a statistically significant behavioral profile had been established.