EsEn

Alvi - AI Virtual Assistant

Sam phone

Brief

Alvi is a passionate fan of Club Atlético Talleres de Córdoba, one of the most beloved and followed football teams in Argentina. This virtual assistant is designed to connect with the club's members and provide them with an interactive and personalized experience. Alvi not only answers questions and resolves concerns but also plays a crucial role in promoting club membership, fostering a sense of community and belonging among fans.

Overview

My role in this project was as a Conversational AI Designer, with a participation of approximately two months.

My responsibilities included mapping topics, designing the bot's personality, and defining its tone and voice, as well as creating and testing the system prompt.

I also conducted workshops to share the knowledge and methodologies developed during the SAM (Santex AI Member) project, including the conversational design stage, system prompt optimization, evaluation methodologies, report structure, relevant metrics, and structuring data sources for better model training.


Goals

Alvi phone
  • Improve Member Experience: Provide quick and accurate responses and an interactive experience that increases satisfaction and loyalty.
  • Promote Club Membership: Facilitate the registration of new members, renew memberships, and inform about exclusive benefits.
  • Foster a Sense of Community and Belonging: Connect members with an assistant that reflects the club's passion and share exclusive content.
  • Optimize Club Communication: Keep members informed about events and activities and facilitate access to social media.

Process

discover
Discover
    define
    Define
      Develop
      Develop
        deliver
        Deliver

          GOALS ACHIEVED

          Alvi is still in the testing phase; however, to measure whether the objectives have been achieved and to evaluate its functionality qualitatively, the following metrics are proposed:

          Alvi phone

          Improve Member Experience:

          • Measure user satisfaction: Implement satisfaction surveys at the end of each interaction and calculate the Net Promoter Score (NPS).
          • Response time: Monitor the average time the assistant takes to respond to member inquiries.
          • First interaction resolution rate: Measure the percentage of inquiries resolved in the first interaction without the need for additional assistance.

          Promote Club Membership:

          • Count new registrations: Track the number of new members registered through the virtual assistant each month.
          • Renewal rate: Measure the percentage of members who renew their membership using the assistant.
          • Conversion of inquiries to memberships: Evaluate the percentage of membership inquiries that result in a new registration.

          Foster a Sense of Community and Belonging:

          • Measure recurring interactions: Count the number of members who interact with the assistant more than once a month.
          • Positive feedback: Collect and analyze positive comments and ratings from users about their sense of belonging to the club.
          • Engagement with exclusive content: Measure the number of views and interactions (likes, shares) with the exclusive content provided by the assistant.

          Optimize Club Communication:

          • Frequency of updates: Monitor the regularity with which news and events are updated in the assistant.
          • Interaction rate with news/events: Measure the percentage of users who access news and events through the assistant.
          • Traffic to social media and website: Count the number of redirects from the assistant to the club's social media and official website.
          Alvi phone
          lessons learned

          Lessons Learned

          • Adjusting a system prompt to regionalisms: Adapting a system prompt to the Argentine dialect is no easy task. It would be beneficial to explore non-generative tools that allow for maintaining consistency in this aspect, ensuring greater accuracy and coherence in the assistant's responses.
          • Importance of structured data sources: Having well-structured data sources is crucial to optimize the training of the LLM. The quality and organization of this data directly influence the model's performance and accuracy.
          • Prior UX research: Conducting prior UX research is essential to define the content users are looking for, rather than focusing solely on what the client wants to show. This improves the bot's effectiveness and ensures a more relevant and satisfying experience for users.
          future improvements

          Future Improvements

          • Integration with appropriate APIs: Implementing integration with APIs that provide real-time data, such as match results and relevant news, to enrich the conversation with the user and keep the information updated.
          • Creation of guidelines: Developing detailed guidelines to establish a clear and efficient process in creating LLM-based virtual assistants, ensuring consistency and quality in design and development.
          • Automated interaction testing: Systematizing the testing process through guided automated interactions is essential to guarantee consistent performance and detect issues early on.
          final thoughts

          Final Thoughts

          • Evaluation of Solutions: Although it is possible to customize LLM-based virtual assistants, it is crucial to assess whether they are the best solution for the specific needs of the project. This evaluation should consider factors such as accuracy, consistency, and the ability to handle regionalisms and other unique aspects of the project.