Copilot’s covered: , ,

Using Copilot Enterprise in a Teams call to summarise the data gave two different responses. The first was me free-form asking it to tell me what we’d talked about. This response seemed okay until the end where there references to customers / projects that weren’t discussed (and I’m not even sure are customers!). The second request was from clicking a built-in prompt. This produced a much more concise and accurate summary.

Takeaway: Microsoft must be applying some massaging in the background with how to format & present back data from it’s curated Copilot prompts.

Background

Copilot is a catch all name for Microsoft’s implementation of Large Language Model (LLM) AI implementations. It is currently based around the Open-AI GPT engines and other associated technologies.

Generative AI is currently a very hot topic and most companies are interested to see how this burgeoning technology can help improve their business processes. An issue we may find with this type of AI is that you have to be careful about what and how you ask it for information to ensure that you get back both accurate, and comprehensive answers. In this style of post I am recording interactions that I’ve had with Copilot to chart and document what types of interactions are successful and which may not be.

Find more in this series here:

Interaction Summary

Technology: Copilot Enterprise inside Microsoft Teams, started during a 1:1 call

Data source: Teams call transcript (~20 minutes of discussion)

Date: Feb 2024

Prompts:

  • Summarise what we talked about
    • free-form prompt
  • Summarize call so far
    • offered prompt, selected

Scenario: Copilot Enterprise in a Teams 1:1 internal call, started after ~ 15 minutes of talking.

Usages: Provided a bulleted list, free-form request wasn’t as structured as the proposed prompt

Outcome:

6/10 for free form prompt

8/10 for suggested/curated prompt

Interaction Record

Replacements in the image are:

  • Yellow – Customer name
  • Blue – Internal persons name
  • Light blue – name of person at customer
  • Orange – “hallucination” customer / project names

Summarise what we talked about
Copilot
Here is a summary of what you talked
about:

  • You discussed the pilot project for
    copilot with <customer>, and the
    tasks and templates you need to
    prepare for it.
  • You leamed about the customer’s
    approach and hypothesis for using
    copilot, and the different user
    groups and personas they have.
  • You reviewed some of the data and
    information that <internal name> has
    gathered about the customer’s
    current state, productivity score,
    and maturity model.
  • You planned to use a employee
    feedback tool or forms to capture
    the subjective feedback from the
    users, and to create some power Bl
    reports to measure the impact of
    copilot.
  • You explored some of the features
    and challenges of loop, and how to
    use it for collaboration and note-
    taking.
  • You also talked about some other
    projects and activities you are
    involved in, such as the invoke
    workshop, the <no idea 1>, and the
    <no idea 2> project.

Al-generated content may be incorrect

Summarize call so far

Copilot

Key Topis:

  • Copilot pilot project with Jan
    Farmer: <internal names> agreed to
    work on some templates, survey
    questions and power Bl reports to
    measure the impact of copilot on
    the users.
  • Hypothesis and user groups:
    <internal names> planned to sync with
    <customer contact>, the customer
    contact, to understand their
    hypothesis and user groups for the
    pilot.
  • Loop tool: <internal names> explored
    the features and limitations of the
    loop tool for collaboration and
    documentation.

Al-generated content may be incorrect


List action items
Suggest follow-up questions