Category: AI

Getting started with Azure Devops MCP in VS Code

What is MCP? It stands for Model Context Protocol and enables your AI Agent (Such as Github Copilot in VS Code) to integrate directly against other systems. One such system is Azure Devops. This tools enables Copilot to read most of your organisation Azure Devops information.

So instead of context switching and copy pasting info from Devops (error messages or User Stories information etc) you can just ask your AI agent directly in your IDE.

Examples of useful prompts for a developer: (from ChatGPT)

Pipeline & Run Inspection

“Find my latest failed run for pipeline Backend-CI and summarize which step failed, including the log excerpt around the error.”

“Get the duration trend for the last 20 runs of ClientSPA-CI and highlight the slowest job.”

“List all pipelines that have not run in the last 60 days.”

“Compare two pipeline runs (Run 1412 and Run 1477) for MyApplication-API and explain the differences in steps, duration, and triggered changes.”

Pull Requests & Repo Info

“Show me all active PRs in the repo MyApplication that have unresolved comments.”

“Which PRs were merged this week, and which work items did they close?”

“Summarize the diff for PR 84303, but only include changes in the Angular project under ClientSPA/.”

“List files in the repo MyApplication that changed in the last 48 hours.”

“Search all repositories for references to ProductRowID.”

Work Items

“Find all work items assigned to me that are blocked by another work item, and summarize the blockers.”

“Show me work items marked as ‘Ready for test’ but not linked to any build or PR.”

“List all bugs created in the last 7 days containing the word paracetamol.”

Artifacts & Variables

“Show me all variable groups used by Backend-CI.”

“Fetch the artifacts for pipeline run 292 of ClientSPA-CI, list their sizes, and highlight anything over 100 MB.”

“Find pipelines that reference the variable AzureAd:ClientSecret.”

 

Setup in VS Code:
azure-devops-mcp/docs/GETTINGSTARTED.md at main · microsoft/azure-devops-mcp

General installation guide:
https://learn.microsoft.com/en-us/azure/devops/mcp-server/mcp-server-overview?view=azure-devops#install-mcp-server

 

What is temperature and top-K parameters used for in context of LLMs?

In the context of large language models (LLMs), temperature and top-K are parameters used to control the randomness and diversity of the generated text. They determine how the model selects words (tokens) when generating responses. Here’s what each one does:

1. Temperature

  • What it does:
    • The temperature parameter adjusts the “creativity” of the model’s outputs. It controls how deterministic or random the token sampling process is.
  • How it works:
    • A low temperature (e.g., 0.2) makes the model more deterministic and focused, favoring tokens with the highest probabilities. This is useful for tasks requiring precision or when predictable outputs are preferred.
    • A high temperature (e.g., 1.0 or more) increases randomness, allowing the model to explore less likely tokens. This is ideal for generating creative or diverse outputs.
    • At temperature = 0, the model selects the highest-probability token every time, making it fully deterministic.
  • Example:
    • Low temperature (0.2):
      Input: “Once upon a time, there was a…”
      Output: “princess who lived in a castle.”
    • High temperature (1.0):
      Input: “Once upon a time, there was a…”
      Output: “dragon guarding a mysterious treasure.”

2. Top-K Sampling

  • What it does:
    • Top-K limits the model’s token selection to the top K highest-probability tokens. This reduces randomness by restricting choices to a smaller, high-probability set.
  • How it works:
    • K = 1: Only the highest-probability token is selected (fully deterministic).
    • Higher K values (e.g., 40, 50): Allow more diverse tokens to be considered, introducing some randomness but still narrowing the choice compared to the full distribution.
  • Why use it:
    • It ensures that only the most relevant or reasonable options are considered, avoiding extremely rare or nonsensical tokens while still allowing variety in outputs.
  • Example:
    • Input: “The sky is…”
    • Top-K (K=1): “blue.”
    • Top-K (K=50): “blue,” “cloudy,” “bright,” “gray,” etc.

When combined:

  • Temperature and top-K can work together to balance creativity and coherence:
    • High temperature + High top-K: Generates diverse and creative text.
    • Low temperature + Low top-K: Generates focused and precise text.
    • Medium temperature + Medium top-K: Balances creativity and reliability.

This combination allows fine-tuning of how the model behaves, depending on the task (e.g., storytelling vs. answering factual questions).