Recommender systems have become indispensable for navigating the vast digital landscape, helping users sift through large volumes of content, products, and services. In the realm of digital health, they play an increasingly vital role in delivering personalized nudges and guidance, enhancing individual engagement, and supporting better health outcomes.
Traditionally, these systems rely on ID-based embeddings, where each user and item (or in our case, each individual and health nudge) is assigned a unique numerical identifier. These IDs are then converted into dense vector representations using techniques like matrix factorization or collaborative filtering. While effective for individuals with interaction histories and popular nudges, this approach has notable limitations, particularly around generalizability and cold-start scenarios.
Enter ID-free embeddings: a promising shift enabled by advances in Large Language Models (LLMs). Instead of relying on arbitrary IDs, these embeddings are generated directly from the rich, descriptive metadata associated with individuals and health nudges—think nudge content (e.g., text, language, tonality), individual health behavior data (e.g., fitness tracker data, dietary logs), demographic data, and health conditions. By leveraging the advanced semantic understanding capabilities of LLMs, ID-free embeddings capture the inherent meaning and relationships within this textual and behavioral information, offering a more flexible and robust foundation for personalized health recommendations.
Why Move Away from ID-Based Embeddings?
While conventional ID-based embeddings have been the mainstay of recommender systems for decades, it has several limitations:
- Cold start: New users or nudges without historical data tend to perform poorly, as the system lacks enough context to generate meaningful recommendations. This leads to a poor initial experience for new users and slower adoption of new nudges.
- Overfitting to history: Models often memorize frequent users or nudges, reducing their ability to generalize.
- Limited portability: These models are transductive in nature, learning about specific users and nudges, and struggle to generalize to new or unseen ones. For example, user 123 in Deployment A of NudgeStream is a distinct individual from user 123 in Deployment B, so the models learned at each deployment cannot be easily transferred across deployments or use cases.
- Operational complexity: ID-based pipelines often require custom retraining and tracking, since user and nudge IDs are unique to each deployment or population.
What is ID-Free Embeddings?
At their core, ID-free embeddings transform the descriptive text and structured data associated with individuals and health nudges into high-dimensional numerical vectors, using a pre-trained LLM encoder. For example, a nudge like “Need a snack? Choose crunchy veggies like carrots to curb hunger without the calories.” Is fed into the LLM, which then outputs a dense vector representing the semantic essence of that nudge. Similarly, an individual’s health profile, including their activity levels from a fitness tracker, dietary preferences, or health goals, can be encoded into an individual embedding. This process allows the recommender system to understand what a nudge is designed to achieve, or who an individual is, based on their attributes and behaviors, rather than just a unique identifier.
In CueZen’s recommender setup:
- User embeddings represent the individual’s current context, derived from a combination of static attributes (e.g. demographics) and dynamic attributes (e.g. recent health behaviors from tracker data).
- Item embeddings represent the nudge content itself. For example, a message like “a gentle reminder to take a 10-minute walk after lunch to boost energy” is encoded into a vector that captures its semantic meaning, such that nudges with similar messages or goals have similar embeddings.
The diagram above illustrates the differences between ID-based and ID-free embeddings. ID-based embedding vectors are created and randomly initialized for every unique individual and nudge. These embeddings are then iteratively updated (along with the model weights) during the training process of the recommender model, such that they optimize some objective or loss function. In the case of ID-free embeddings, user and nudge metadata are transformed into semantic embeddings via a pre-trained LLM, using only semantic content and not requiring any IDs. Model training proceeds as usual, where model weights are iteratively updated to optimize the objective or loss function.
Benefits of ID-Free Embeddings
This semantic approach offers several advantages for building robust and effective digital health recommender systems:
- Eliminating cold-start scenarios: Semantic embeddings can be computed for new users or nudges and directly plugged into a recommender model without requiring interaction history [1]. This means relevant recommendations can be made even for brand new users or nudges right out of the gate, leading to a much more engaging initial experience.
- Enhanced Generalization and Universal Representations: ID-free embeddings provide universal representations that generalize across different populations and deployments. For example, two individuals from different countries—both 35 years old, prediabetic, overweight, and sedentary—would have similar embeddings. Similarly, two nudges containing the same topic and advice would have similar embeddings even if they were written in different languages. This allows models to learn patterns from the underlying behavior and content, leading to strong performance on previously unseen users or nudges and mitigating the problem of overfitting to historical data [2].
- Improved Transferability and Simplified Deployment: Because these embeddings capture intrinsic meaning rather than arbitrary IDs, models trained on one population and use case can transfer effectively to others, delivering good results from day one [3]. Fine-tuning on the target population can further enhance recommendation performance. This also simplifies model deployment, as precomputed embeddings remove the need for ID-specific retraining or the tracking of large ID-based embedding matrices across different deployments.
- Reusability of Existing Recommender Architectures: A practical benefit is that many existing recommender architectures can be directly reused, by simply replacing the randomly initialized ID-based embedding matrix with precomputed semantic embeddings. ID-free embeddings can be integrated seamlessly into the vast body of existing recommender models and proven architectures, without needing to reinvent the wheel.
- Privacy-Aware Design: By reducing dependence on personal identifiers and focusing on semantic attributes, this approach inherently supports safer handling of sensitive customer health data.
Related Work
Recommender systems using ID-free embeddings are a rapidly evolving area of research. Built on the strong foundations of “traditional” recommender systems, the use of LLMs opens up many new possibilities in addressing previous limitations such as cold start, transferability and generalization across domains. As a result, researchers are increasingly interested in ID-free or modality-based approaches, which leverage semantic embeddings derived from content or behavioral signals.
The foundation for semantic retrieval models was laid by early neural recommender systems like DSSM [4] and the Youtube DNN model [5], which introduced architectures to map users and items into shared embedding spaces using behavioral and content features. Further advancements, such as Two-tower models [6] and attention-based architectures like DIN [7] and DIEN [8], significantly enhanced the ability to model user intent without relying solely on explicit ID representations.
Recent progress has pushed this direction further by demonstrating strong performance using purely semantic inputs. For instance, large language models (LLMs) have achieved state-of-the-art results for collaborative filtering using only textual item descriptions [9]. Similarly, Recformer [10] learns sequential user preferences by applying language modeling techniques directly to item content, thereby bypassing traditional ID lookups entirely.
Zero-shot and transferrable recommendation is another emerging area. ZESRec [11] performs well on cold-start tasks without requiring historical user data, while universal representation learning (UniSRec) [12] and vector-quantized embeddings (VQ-Rec) [13], demonstrate improved generalization across users and domains.
From a broader perspective, [14] present a comparative study between ID-based and modality-based recommenders, concluding that semantic models are not only viable alternatives but, in some cases, superior—especially in sparse or evolving environments. These findings align with our motivation to adopt ID-free recommendation strategies based on behavioral and content embeddings.
What’s Next?
In the next few posts, we’ll go deeper into the following:
- Comparing our existing ID-based Knowledge Graph Attention Network (KGAT) model with new ID-free approaches.
- Evaluating performance of the ID-free models across varying training data sizes.
- Additional ablation studies and evaluations on the robustness of semantic embeddings.
These explorations will help us assess the practical value of ID-free models and guide our next steps in improving our nudge engine.
References:
[1] S. Sanner and e. al., “Large Language Models are Competitive Near Cold-start Recommenders for Language- and Item-based Preferences,” in RecSys, 2023.
[2] K. Zhang and e. al., “Learning ID-free Item Representation with Token Crossing for Multimodal Recommendation,” arXiv, 2024.
[3] Y. Li and e. al., “A Zero-Shot Generalization Framework for LLM-Driven Cross-Domain Sequential Recommendation,” arXiv, 2025.
[4] P.-S. Huang and e. al., “Learning deep structured semantic models for web search using clickthrough data,” in CIKM, 2013.
[5] P. Covington and e. al., “Deep Neural Networks for YouTube Recommendations,” in RecSys, 2016.
[6] X. Yi and e. al., “Sampling-bias-corrected neural modeling for large corpus item recommendations,” in RecSys, 2019.
[7] G. Zhou and e. al, “Deep Interest Network for Click-Through Rate Prediction,” in KDD, 2018.
[8] G. Zhou and e. al., “Deep interest evolution network for click-through rate prediction,” in AAAI, 2019.
[9] R. Li and e. al., “Exploring the Upper Limits of Text-Based Collaborative Filtering Using Large Language Models: Discoveries and Insights,” arXiv, 2023.
[10] J. Li and e. al., “Text Is All You Need: Learning Language Representations for Sequential Recommendation,” arXiv, 2023.
[11] H. Ding and e. al., “Zero-Shot Recommender Systems,” Amazon Scienc, 2021.
[12] Y. Hou and e. al., “Towards Universal Sequence Representation Learning for Recommender Systems,” in KDD, 2022.
[13] Y. Hou and e. al., “Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders,” in WWW, 2023.
[14] Z. Yuan and e. al., “Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisited,” arXiv, 2023.
