AI Risk Management Training for Executives: Put a Number on It
Ask a security leadership team to rank "prompt injection in the corporate AI assistant" against "ransomware on the file servers" for next year's budget, and most will answer with a color. Amber, probably. Then the CFO asks how much amber costs, and the conversation stalls.
That stall is the gap that AI risk management training for security executives should close. Most courses spend the day on frameworks (NIST AI RMF, ISO/IEC 42001, the EU AI Act) and send leaders home able to classify a risk but unable to price it. Classification decides whether something goes on the register. Price decides whether it gets money. If a course does not teach the second skill, the register entry sits in amber forever.
Why the heat map fails for AI risk
The problem with risk matrices is not new. Louis Anthony Cox Jr. laid it out in "What's Wrong with Risk Matrices?" (Risk Analysis, 2008): coarse likelihood and impact bins can assign identical ratings to risks that differ by orders of magnitude, and in some cases rank a smaller risk above a larger one. Douglas Hubbard and Richard Seiersen made the security-specific version of that argument in How to Measure Anything in Cybersecurity Risk.
AI makes it worse. "Likelihood: medium" for prompt injection hides the one number you can actually measure, which is how often an attack succeeds against your deployment. Throwing that measurement away to pick a bin is a bad trade.
Decompose one scenario with FAIR
The Open Group's Open FAIR standards (O-RT for the taxonomy, O-RA for the analysis) split risk into loss event frequency and loss magnitude. Loss event frequency is threat event frequency times susceptibility: how often an attacker tries, and what fraction of tries succeed.
Take one concrete scenario. An external sender plants instructions in an email or shared document, your assistant retrieves it while answering an unrelated question, and the model leaks internal data through a rendered link. MITRE ATLAS calls the technique [AML.T0051.001, indirect prompt injection](/atlas/AML.T0051.001), and the outcome [AML.T0057, LLM Data Leakage](/atlas/AML.T0057). EchoLeak (CVE-2025-32711) in Microsoft 365 Copilot followed this exact path; our [AI incident tabletop post](/blog/executive-ai-risk-training-tabletop) walks through it as an exercise.
Three inputs, and they come from three different places:
- Attempt frequency (estimated). How many crafted payloads reach the retrieval index per year. You will not know this. Give a calibrated 90 percent range instead, say 2 to 20, and write down why.
- Susceptibility (measured). The fraction of attempts that succeed. This is where AI risk differs from most FAIR work: your red team can run the attack 60 times against your own tenant with PyRIT or garak and count. Seven successes in 60 trials is evidence. "The vendor says it is mitigated" is not.
- Loss per event (estimated with finance and legal). Response hours, notification, counsel, regulatory exposure. For a sense of the upper end, IBM's Cost of a Data Breach Report 2025 puts the global average breach at US$4.44 million, and GDPR Article 83 allows fines up to 4 percent of worldwide annual turnover. A range of $50,000 to $2 million per leak event is a defensible starting point for most mid-sized firms; yours may differ.
The simulation
About twenty lines of NumPy turn those inputs into a loss distribution. Ranges become lognormals, red team counts become a Beta distribution, and attempts become Poisson draws.
import numpy as np
rng = np.random.default_rng(7)
N = 100_000
def lognormal_from_ci(low, high, size):
# treat (low, high) as a calibrated 90% interval
mu = (np.log(low) + np.log(high)) / 2
sigma = (np.log(high) - np.log(low)) / (2 * 1.645)
return rng.lognormal(mu, sigma, size)
def simulate(successes, trials):
attempts = rng.poisson(lognormal_from_ci(2, 20, N)) # payloads reaching retrieval per year
p_success = rng.beta(1 + successes, 1 + trials - successes, N) # red team result
events = rng.binomial(attempts, p_success) # loss events per year
return np.array([lognormal_from_ci(50_000, 2_000_000, k).sum() for k in events])
before = simulate(successes=7, trials=60) # current config
after = simulate(successes=0, trials=60) # external mail excluded from retrieval, link allowlist on
for name, ale in [("before", before), ("after", after)]:
print(f"{name}: mean ${ale.mean():,.0f} P90 ${np.percentile(ale, 90):,.0f} "
f"P(loss > $1M) {np.mean(ale > 1e6):.1%}")
You can sanity-check the means without running anything. The 2 to 20 attempt range has a mean of about 8. The Beta posterior for 7 of 60 has a mean of 8/62, about 13 percent. The $50,000 to $2 million loss range has a mean near $590,000. Multiply them: roughly $620,000 a year in expected loss before the fix. After the fix, the success rate drops to a posterior mean of 1/62, and expected loss falls to about $80,000.
That difference, around $540,000 a year, is the number to put next to the cost of the control. It is also the number that tells you the control does not get you to zero. Zero successes in 60 trials still leaves a 95 percent upper bound of about 5 percent (the rule of three: 3/n), and the simulation carries that uncertainty instead of rounding it away.
The P90 and P(loss > $1M) lines matter more than the mean for a board. "One year in ten, this scenario costs us more than X" is a sentence directors can act on. NIST's AI RMF asks for this work without naming a method: MAP 5.1 covers documenting likelihood and magnitude of impacts, and MEASURE 2.7 covers evaluating security and resilience. A script an auditor can rerun meets both. If you want a packaged version, pyfair implements the FAIR ontology in Python.
What a course should make you do
Judge any AI risk management course by one exercise: does each participant leave having priced a scenario from their own environment, with at least one measured input? Lectures on likelihood scales do not count. Neither does a vendor's risk calculator with the inputs pre-filled.
The measured input is the part executives tend to skip, because it requires asking the red team for trial counts instead of a pass/fail verdict. Our post on [counting trials in AI red team work](/blog/ai-red-team-training-security-engineers) explains what to request. Calibration is the other piece: Hubbard's equivalent-bet test (would you rather bet on your range containing the answer, or on a spin with a 90 percent chance of winning?) takes twenty minutes to teach and fixes most overconfident ranges.
Where quantification breaks
It needs an inventory first. You cannot price a scenario for an assistant nobody knew was connected to the mail server; our [CISO post](/blog/where-cisos-learn-ai-security) covers finding those through proxy logs. It also decays. A measured success rate describes one model version under one system prompt, and SaaS vendors swap models without notice, so the red team rerun belongs on a schedule (quarterly, and after any vendor model change). And it handles the tail poorly. A regulator deciding to make an example of the first AI leak in your sector is not well described by a lognormal; say that out loud rather than letting the P90 imply more precision than it has.
None of this replaces judgment. It gives judgment something to argue with.
GTK Cyber's [A Cyber Executive's Guide for Artificial Intelligence](/courses/executive-ai-guide) covers AI risk and governance frameworks and ROI evaluation for AI security tools, and we deliver it privately for leadership teams through an [executive briefing](/executive-briefing).
Azure API Management Developer Portal and API Versioning: Version Sets Without Breaking Callers
How do you ship a breaking change to the order API without breaking the partners who already call it, when the next partner should find the API in the developer portal, read the docs and get a key without opening a ticket? The versioning half has a trap that decides the rest: API Management fixes the versioning scheme for every version in a version set the moment you add the first one, and the live API keeps its URL only if it stays the Original version. Give it a v1 identifier and every existing call stops matching. The short answer for the order API: path versioning, the live API kept as Original, v2 only for the breaking change, revisions for everything else, and two products so a free tier and partners get different limits and only partners need approval.
After Part 3 the order API is locked down and nobody can find it. There is one product, one subscription that Bicep created for a partner called Contoso Shop, and an Entra ID token check in front of every operation. A new partner learns the API exists from an email, gets the OpenAPI file as an attachment and waits for someone to create a key. The companion repo now enables the developer portal, adds a global CORS policy so the portal's test console can reach the gateway, puts the existing API into a path-based version set next to a v2 with a paged list-orders, and splits access into an auto-approved free product and a partner product that needs approval. The instance is still Basic v2, and that has two consequences: the portal is off until you turn it on, and the managed portal can't host custom widgets. The Bicep and the function project build clean; none of it has been deployed yet, so where the article depends on how a live instance behaves, it says so and points to the check in the repo's scripts/verify.sh.
What the developer portal builds from your OpenAPI file
The portal's API reference is the OpenAPI file you imported in Part 1, rendered. This is list-orders in openapi/orders.yaml:
paths:
/orders:
get:
operationId: list-orders
summary: List orders
description: Returns all orders, newest first. Filter with `status`.
parameters:
- name: status
in: query
required: false
description: Only orders in this status.
schema:
$ref: '#/components/schemas/OrderStatus'
responses:
'200':
description: Orders, newest first.
content:
application/json:
schema:
type: array
items:
$ref: '#/components/schemas/Order'
Learn calls the portal "an automatically generated, fully customizable website with the documentation of your APIs", and "out of the box, the developer portal is already populated with your published APIs and products" (overview). For the order API that means a list of APIs, a reference page per operation built from the imported definition (method, URL, parameters, request and response schemas), a products page, sign-up and sign-in, and a profile page where a signed-in developer sees their subscription keys. Nobody writes these pages; they change when the API definition changes.
Three defaults decide what a visitor gets:
- Only published products show up. A product's state: 'published' makes it discoverable in the portal, and that's all it does. Learn is explicit that the gateway doesn't care: "Regardless of whether the product is published, all of the product's APIs are accessible for requests via the API Management gateway" (products). Unpublishing a product hides it from new developers; it doesn't revoke a single key.
- The profile page only lists product keys. "Currently, the developer portal only shows the product scope subscriptions under the User Profile section" (subscriptions). A subscription scoped to one API or to all APIs works at the gateway and is invisible to its owner in the portal. If developers are meant to manage their own keys, scope every subscription to a product.
- Anyone can read it. "By default, the developer portal enables anonymous access." Anonymous visitors are in the built-in Guests group and see the pages and the products made visible to Guests, until you switch on the site setting that requires sign-in. For many partner APIs that's fine, since the docs aren't the secret; the keys are. If the API's shape is itself confidential, keep its products away from Guests and require sign-in before the first publish, not after.
Sign-up uses API Management's own user accounts by default; Entra ID and Entra External ID sign-in are the alternatives (not on the classic Basic tier or Consumption). A user who signs up lands in the built-in Developers group, and which products a developer can see is decided per group. The products section below uses that to keep the partner product out of sight for everyone outside a partners group.
Turning it on in Basic v2
On the classic tiers the portal exists from the moment the instance does. On v2 tiers it doesn't: "If you created your instance in a v2 service tier, first enable the developer portal" (customize). In the Azure portal that's Developer portal > Portal settings > Enabled > Save, which "might take a few minutes". In Bicep it's one property on the service in infra/apim.bicep:
properties: {
publisherEmail: publisherEmail
publisherName: 'Orders demo'
// v2 tiers create the instance with the portal off; Consumption has no portal at all.
developerPortalStatus: apimSku == 'Consumption' ? null : 'Enabled'
}
developerPortalStatus takes 'Enabled' or 'Disabled' in resource API version 2024-05-01, which the sample already uses (template reference). The sample's apimSku parameter still allows Consumption, and Consumption is the one tier without a managed portal, so the property is null there instead of asking for something the tier can't do.
Learn says the enable step works on every v2 tier. One Microsoft Q&A answer (not documentation) reports an instance where the option was missing on Basic v2. This sample hasn't been deployed with the property yet, so treat Learn as the authority and check your own instance: after the deployment, Portal overview in the Azure portal should show the portal URL, https://<service name>.developer.azure-api.net.
Enabling the portal doesn't make it visible. The portal content has to be published, and the first publish is a manual step: open Portal overview, select Developer portal to open it in the administrative interface, then Publish. The same rule applies to every later change, because "Saved changes are visible only to you and aren't visible to portal visitors until you publish them." Learn also lists service changes that need a republish: a custom domain, identity providers, delegation, sign-in and product terms. Changes to an API definition are not on that list. That fits Learn's split between portal content (pages, styles, layouts) and API Management data (APIs, operations, products), but it's an inference; republish once after the first import and check. No Learn page shows publishing from Bicep, so the sample leaves it as a README step.
Descriptions and examples that end up on the page
Learn doesn't enumerate which OpenAPI fields the reference page renders, so the sample fills in the ones a partner would look for, and you check the result on your instance before relying on any of them. The order spec now carries three kinds of text:
- an operation description that says what the call returns and how to filter it,
- a description on each parameter, which is where a partner looks to find out what status filters on,
- example values on schema properties, from components.schemas in the same file:
required: [sku, quantity, unitPrice]
properties:
sku:
type: string
example: BOOK-0042
quantity:
type: integer
minimum: 1
example: 2
unitPrice:
type: number
format: decimal
example: 24.90
Write these for the person reading the portal, not for the code generator. Only orders in this status. is short, but it answers the question a partner has at that spot; a description that repeats the parameter name in a sentence answers nothing. The examples matter more than they look: a developer copies them into their first request, so keep the set consistent (the same BOOK-0042 everywhere).
Two more texts reach the portal without touching the spec. When you make a revision current, the optional change log "is included in the developer portal for your API users to view" (revisions), which is the only release-notes channel the portal has built in. And each API version carries its own spec, so the v2 reference page documents the page envelope while the Original keeps documenting the array; the versioning section below covers how the two coexist.
The try-it console and the CORS error everyone hits
Every operation page has a Try it button. It opens a console that fills in the request URL, takes a subscription key (a signed-in developer can pick one of their own), lets you set parameters and headers, and sends the call. Learn lists three authorization models for it: no authorization, a subscription key, and OAuth 2.0, where "you can configure the test console to generate a valid OAuth token on behalf of the test console user".
The first time someone selects Send against a fresh instance, the browser shows a CORS error and nothing reaches the API. The console is JavaScript running on https://<service name>.developer.azure-api.net and calling https://<service name>.azure-api.net, a different origin. The subscription key travels in the Ocp-Apim-Subscription-Key header, a custom header, so the browser first sends an OPTIONS preflight asking whether the portal's origin may send it. Without a cors policy the gateway's answer carries no Access-Control-Allow-Origin, and the browser stops there. Learn: "If the developer portal's domain name isn't an allowed origin for cross-domain API requests, test console users will see a CORS error" (enable CORS).
The fix is a cors policy at the top of the global policy, infra/policies/global.xml:
<inbound>
<cors allow-credentials="false">
<allowed-origins>
<origin>{{portal-origin}}</origin>
</allowed-origins>
<allowed-methods preflight-result-max-age="300">
<method>*</method>
</allowed-methods>
<allowed-headers>
<header>*</header>
</allowed-headers>
<expose-headers>
<header>*</header>
</expose-headers>
</cors>
The origin comes from a named value, built in apim.bicep from the instance name and added to the named values the policies already read:
// The default portal host; add a custom portal domain here too, or the try-it console gets a CORS error there.
var portalOrigin = 'https://${apim.name}.developer.azure-api.net'
// Plain (non-secret) values the policies read as {{name}}: IDs and the portal origin, not credentials.
var policyNamedValues = {
'tenant-id': tenant().tenantId
'orders-api-app-id': ordersApiAppId
'orders-backend-app-id': backendAppId
'orders-test-client-app-id': testClientAppId
'portal-origin': portalOrigin
}
What each part of the policy decides:
- One origin, not *. The cors policy reference wants a full URI with scheme and host for each origin, and warns that * "may be overly permissive". Allowing every origin would let any website call your API from a visitor's browser with whatever key that page holds. With a custom domain on the portal, add its origin as a second <origin>; the custom domain setup can add it for you.
- allow-credentials="false". The console sends the key and, with OAuth, the token as headers. It has no cookies to send to the gateway, so there is no reason to allow them. This is the policy's default; it's spelled out so nobody flips it to true while debugging.
- Methods and headers *. The console sends whatever the operation needs: POST with Content-Type for create-order, the key header, Authorization. Listing them one by one is tighter, and it's also the list you forget to update when the next API arrives on the same instance.
- At the top of inbound, before everything else. Learn's troubleshooting list warns of "unexpected behavior if the cors policy is not the first policy in the inbound section". It also documents why the Part 3 token check doesn't break the preflight: "Only the cors policy is evaluated on the OPTIONS request during preflight. Remaining configured policies are evaluated on the approved request." The OPTIONS call carries no key and no token, and it doesn't need to.
Global, not per API, and only once. The portal's own Enable CORS button on Portal overview writes the same kind of policy at the global scope, for all APIs, and that's the right place for it: the v2 API and every later API inherit it. Take the warning on Learn's portal page literally: "Only one CORS policy is executed. If you specify multiple CORS policies (for example, on the API level and on the all-APIs level), your interactive console may not work as expected." The cors reference says you may configure it at more than one scope, which reads like a contradiction until you note that "generally, only the first cors policy is applied". Keep one, globally, and you never have to work out which one won.
Never on the product. Putting the policy on orders-partners looks tidy, since only partners should use the console. It doesn't work: "If you apply the CORS policy in the Product scope, instead of the API(s) scope, and your API uses subscription key authentication through a header, your console won't work." The preflight carries no subscription key, so API Management can't tell which product the call belongs to, and a product-scoped policy never runs for it. Learn's workaround is the key in the query string, which Part 1 argued against: the key ends up in every access log between the browser and the gateway. Move the policy to global scope instead.
When the gateway itself isn't reachable from the browser (a network-isolated instance), the console can route its calls through the portal's backend instead: the Operation: Details widget has a Use CORS proxy setting, and after saving you republish the portal. That needs no cors policy at all, at the price of the portal backend making the calls for you.
scripts/verify.sh sends both preflights after a deployment (cases P4 and P5): one from the portal origin, which should come back 200 with Access-Control-Allow-Origin, and one from https://example.com, which should get an empty 200 without that header, the policy's documented behaviour for an origin it doesn't match.
The Entra token the console doesn't have
With CORS fixed, the order API's console call still fails, now with a 401. Part 3 put validate-azure-ad-token on the API, and the console sends a subscription key and nothing else. That's the same call as case V2 in the verify script: key, no token, rejected at the gateway. The v2 API from the versioning section reuses the same API policy file, on purpose: a new version shouldn't be a way around the token check.
The console can get a token, through OAuth 2.0 user authorization (Learn). You register a second Entra app that represents the console, add an OAuth 2.0 authorization server to API Management pointing at Entra ID, and select that server in the API's settings. The console then shows an authorization drop-down, signs the developer in with the authorization code flow and adds the Authorization header to the call. As Part 3 quoted from the same page, this "only enables the developer portal's test console (and the test console in the Azure portal) as a client to acquire a token"; validating the token stays the policy's job.
Even with OAuth set up, Part 3's policy would still reject the console's token, because it was written for partner backends that call as themselves.
Two checks get in the way. Part 3's policy pins the calling app in client-application-ids to the test client, and the console's token is issued to the console's own app, so its azp is the wrong app ID. The policy also requires Orders.ReadWrite in roles, and that role is defined with allowedMemberTypes: ['Application'], so no signed-in developer can be assigned it.
The console keeps getting a 401 until the policy accepts the console app's ID and the role (or a separate one) allows user assignments. That follows from the claims Part 3 walked through, not from a Learn page about the console, and it hasn't been tested here.
This article doesn't show the console calling the order API. Every step above needs at least one more app registration, and the tenant the sample is built in doesn't allow users to create them, the same constraint that kept Part 3's token checks a runbook. With a subscription key alone, the console works for any API that doesn't check a token. For the order API, the options are OAuth configured as above, or reference pages whose console can't complete a call to it.
Versioning: one scheme per version set, chosen once
"Turning on versioning" in API Management means creating a version set, and a version set holds very little: a display name and a scheme. The versions themselves are ordinary APIs. "Each version of an API is maintained as its own API resource and is associated with a version set" (versions), so v2 of the order API isn't a flag on the existing API. It's a second API resource with its own spec, operations and policies that shares a path and a portal entry with the first.
The scheme is the part you can't take back. "All versions in a version set have the same versioning scheme. It's based on the versioning scheme that's used when you first add a version to an API." There is no migration from header to path later. A different scheme means a new version set, and a new version set means every caller changes how they ask for a version.
Path: the version in every log line
The sample's version set in infra/apim.bicep. ARM calls path versioning Segment:
// Path versioning (Segment in ARM), fixed for every version in the set by the first one.
resource ordersVersionSet 'Microsoft.ApiManagement/service/apiVersionSets@2024-05-01' = {
parent: apim
name: 'orders'
properties: {
displayName: 'Orders API'
versioningScheme: 'Segment'
}
}
The existing API from Part 1 joins it with one new property:
// The Original version: in the set, but no apiVersion, so existing callers keep /sales/orders.
// Giving it apiVersion: 'v1' would move it to /sales/v1/orders and break every existing call.
resource ordersApi 'Microsoft.ApiManagement/service/apis@2024-05-01' = {
parent: apim
name: 'orders'
properties: {
displayName: 'Orders API'
apiVersionSetId: ordersVersionSet.id
path: 'sales'
protocols: ['https']
format: 'openapi'
value: loadTextContent('../openapi/orders.yaml')
serviceUrl: functionBaseUrl
subscriptionRequired: true
}
}
There's deliberately no apiVersion line. Without it, the API is the Original: "If you add a version to a non-versioned API, an Original version will be automatically created and will respond on the default URL, without a version identifier specified. The Original version ensures that any existing callers aren't affected." /sales/orders keeps answering, and every partner integration keeps working without a release on their side.
Adding apiVersion: 'v1' next to v2 looks tidier, and it's the trap from the introduction. An API with an identifier answers only on its versioned URL, so the live API moves to /sales/v1/orders. The deployment is green, and every existing call to /sales/orders now matches no API at all. Learn doesn't document the status code for a call that matches no version, so verify.sh case P3 calls /sales/v9/orders and prints whatever comes back rather than asserting a number.
Learn describes the Original for the Azure portal's flow, where the existing API is linked to the set "but a version identifier doesn't need to be specified". It doesn't show the ARM form. Leaving apiVersion out is the direct translation, and the Bicep builds, but whether a deployed instance keeps the Original on its old URL is what case P1 checks, and P1 hasn't run. If ARM insists on an identifier, the README's fallback is v1, with the URL change announced to partners before the deploy rather than discovered by them after it.
v2 is the second API resource:
// v2 breaks one contract: list-orders returns a page envelope. Same path, identifier in the URL: /sales/v2/orders.
resource ordersApiV2 'Microsoft.ApiManagement/service/apis@2024-05-01' = {
parent: apim
name: 'orders-v2'
properties: {
displayName: 'Orders API'
apiVersionSetId: ordersVersionSet.id
apiVersion: 'v2'
apiVersionDescription: 'list-orders returns a page of orders ({ items, nextLink }) instead of an array.'
path: 'sales'
protocols: ['https']
format: 'openapi'
value: loadTextContent('../openapi/orders-v2.yaml')
serviceUrl: functionBaseUrl
subscriptionRequired: true
}
// Two APIs on one path only work inside a version set, so the Original has to join it first.
dependsOn: [ordersApi]
}
Same path, same version set, identifier v2: the gateway serves it at /sales/v2/orders. The dependsOn exists because two APIs may share sales only as members of one version set. ARM would otherwise be free to create orders-v2 while the Original is still a standalone API on that path.
The breaking change lives in openapi/orders-v2.yaml: list-orders returns { "items": [...], "nextLink": "orders?skip=50" } instead of a bare array. Every client that deserialises the response into a list breaks on that, which is what makes it a version and not a revision. The function app gets one new function, list-orders-v2 on route v2/orders, and the v2 operation gets a policy that sends it there:
<inbound>
<base />
<rewrite-uri template="/v2/orders" copy-unmatched-params="true" />
</inbound>
copy-unmatched-params carries status and skip over to the rewritten URL. Everything else in v2 (the other two operations, the API policy, the token check) is the same as in the Original.
Path versioning costs you clean resource URLs: /sales/v2/orders and /sales/orders are the same orders. In return the version is in every access log line, every bug report that pastes a URL, and every curl a partner sends to support. For server-to-server partners on mixed stacks, I'd take that visibility over clean URLs.
Header: clean URLs, invisible version
The sample doesn't use it, but the version set would differ in two properties (apiVersionSets reference):
properties: {
displayName: 'Orders API'
versioningScheme: 'Header'
versionHeaderName: 'Api-Version'
}
Every caller now sends Api-Version: v2, and the URL stays /sales/orders for all versions. The cost moves to the client side, where you can't see it:
- A forgotten header isn't an error. A request without the header is a request without an identifier, and with an Original in the set that's the Original's URL. A partner who upgrades their client code to the v2 response shape but misses the header gets the old array back, and the bug surfaces as a deserialisation failure in their code, not at your gateway.
- The version disappears from logs. Access logs and pasted URLs no longer say which version was called. You get it back only by logging the header explicitly.
- Every hop has to keep the header. A curl needs -H, a browser address bar can't send it at all, and any proxy or HTTP cache between partner and gateway has to pass the header through and key on it. A cache that keys on the URL alone needs Vary: Api-Version, or it serves one version's response to another version's caller. That's general HTTP caching behaviour, not something API Management documents.
The try-it console needs one more thing. The version header is a custom header, so it triggers a preflight, and the cors policy has to allow it. The sample's global policy allows *, which covers it; if you tighten allowed-headers to a list, add the version header to it. That follows from how CORS works and isn't stated on Learn. What Learn does state is the scope rule: "If you configure the cors policy at the API scope, and your API uses a header-versioning scheme, the policy won't work because the version is passed in a header" (cors policy). The preflight doesn't carry the version header's value, so the gateway can't pick the version whose policy should answer. It's the product-scope problem again, with a header in place of the key, and the global policy from the CORS section avoids both.
Query string: the parameter that drops out of your spec
versioningScheme: 'Query' with versionQueryName: 'api-version' gives /sales/orders?api-version=v2. It's Azure's own convention (the az rest calls later in this article send ?api-version=2024-05-01 to ARM), it shows up in logs, and it works from an address bar.
This one bites in the tooling rather than at the gateway. Learn: "Query parameters aren't allowed in the servers property of an OpenAPI specification. If you export an OpenAPI specification from an API version, a query string won't appear in the server URL." A partner who downloads the v2 spec from the portal and generates a client from it gets a client that never sends api-version, which means it calls the Original and receives the v1 contract. The same goes for anything else that treats the exported file as the contract, such as a contract test or a mock server. If you pick query versioning, document the parameter in each operation of the spec yourself, and don't assume an exported definition round-trips.
Versions or revisions
Most changes to the order API shouldn't be a version at all. Learn's rule of thumb: "versions are used to separate API versions that have breaking changes, and revisions can be used for minor and non-breaking changes" (revisions). A new optional field, a new operation, a policy fix: those are revisions. A revision is a numbered copy of the same API that you edit and test without callers noticing, at https://<service name>.azure-api.net/sales;rev=2/orders (the ;rev=n goes after the API path, before the operation). Making it current is the release, and the optional change log from the portal section is the release note.
The mistake to watch for is the reverse: a breaking change shipped as a revision. "It's only a field rename" goes out as rev=3, and every partner gets it at the moment someone selects Make current, with no URL to opt out on. If a revision turns out to be breaking while it's still being tested, the Azure portal can turn it into a version with Create Version from this Revision, which keeps the work and puts the change behind its own identifier. Some properties can't change in a non-current revision at all (name, path, protocols, subscription required, API version among them), so path and scheme changes are version-level decisions anyway.
The four options side by side:
!Path, header and query-string versioning compared with a revision: what the caller sends, log visibility, OpenAPI export, CORS, caching and the main risk
Picking by situation:
- External partners, many client stacks, support by email: path, with the live API as the Original. That's the order API's choice.
- Clients you build and ship yourself, URLs stored where you can't easily change them: header, and keep an Original so a missing header has a defined answer.
- An API that already follows the Azure-style api-version convention: query string, with the parameter written into the spec by hand.
- Anything a correct client wouldn't notice: a revision, whichever scheme the set uses.
Products as subscription tiers
Part 1 put the order API into one product and Part 2 gave that product a rate limit. A portal with self-service sign-up needs a second one, because the developer who signs up on a Sunday to try the API shouldn't get the same key a contracted partner gets. Both products contain the same APIs; the differences are approval, the subscription limit and the policy.
The free tier in infra/apim.bicep:
// Evaluation: auto-approved, one subscription per user account, low limits.
resource freeProduct 'Microsoft.ApiManagement/service/products@2024-05-01' = {
parent: apim
name: 'orders-free'
properties: {
displayName: 'Orders (free)'
description: 'Try the order API: 5 calls a minute, 1,000 a week. Keys are issued right away.'
subscriptionRequired: true
approvalRequired: false
subscriptionsLimit: 1
state: 'published'
}
}
The partner product is the Part 1 resource with approvalRequired switched to true. Their policies, product-orders-free.xml and then product-orders-partners.xml, differ only in the inbound section:
<inbound>
<base />
<rate-limit calls="5" renewal-period="60" />
<quota calls="1000" renewal-period="604800" />
</inbound>
<inbound>
<base />
<rate-limit calls="60" renewal-period="60" />
</inbound>
The free tier has two limits because they stop different things. rate-limit caps bursts: the sixth call inside a minute gets a 429 (verify.sh case P6). quota caps volume: 1,000 calls in 604,800 seconds, a week, after which "the caller receives a 403 Forbidden response status code" with a Retry-After header (quota). Five calls a minute without a quota still allows about 50,000 calls a week from a key nobody reviewed. Two details of quota matter here: it's allowed only at product scope, and the week starts "relative to the start time of the subscription", so every free key has its own week and no Monday-morning reset. The partner product's 60 calls a minute raises Part 2's 10. As Part 2 showed, product policies only run for product-scoped keys; the built-in all-access key the Azure portal's test tab uses skips both limits.
subscriptionsLimit: 1 means one free subscription per user account. It isn't one per person: nothing stops someone from signing up with five email addresses and calling with five keys at 25 calls a minute. The free tier's numbers are set low enough that this buys little, and the partner product, where the real throughput is, sits behind approval. The 2024-05-01 products template reference also says the property "can be present only if subscriptionRequired property is present and has a value of false", which contradicts the portal tutorial and the sample's protected product. The same reference for older API versions such as 2019-12-01 says "a value of true", so the reference versions disagree; the Bicep builds, and it's unverified until a deployment accepts it.
Since each version is its own API, each version goes into each product:
// Every version is its own API, so each one goes into each product.
var orderApiVersions = [ordersApi.name, ordersApiV2.name]
resource partnerProductApis 'Microsoft.ApiManagement/service/products/apis@2024-05-01' = [for api in orderApiVersions: {
parent: partnerProduct
name: api
}]
resource freeProductApis 'Microsoft.ApiManagement/service/products/apis@2024-05-01' = [for api in orderApiVersions: {
parent: freeProduct
name: api
}]
Forget this for v2 and it's in no product: "To publish the API, you must associate it with a product" (publish versions). No product key can call it, and the portal has no product to show it in.
Who sees which product is decided by groups: "Products are first made visible to groups, and then developers in those groups can view and subscribe to the products that are associated with the groups" (groups). The sample makes the free product visible to the built-in developers group (everyone who signed in) and the partner product visible to a custom partners group, both through products/groups resources. Neither is visible to Guests, the group anonymous visitors fall into.
Approving a partner subscription
A partner's path through the portal, with the sample's settings:
1. They sign up and land in Developers. They see the free product and can take a key straight away.
2. Someone on the API team adds their account to partners (Azure portal > the instance > Users). Only now does Orders for partners appear for them.
3. They request it. Because the product requires approval, the new subscription is created in the submitted state, which the subscriptions template reference defines as "request has been made by the developer, but has not yet been approved or rejected".
4. API Management sends the Subscription requests (requiring approval) notification to whoever is configured for it under Notifications on the instance (notifications). If nobody is configured, nobody gets an email and the request just sits there.
5. Someone approves it, in the Azure portal under Subscriptions or from a script.
The script version, from the README, lists the waiting subscriptions and activates one:
APIM=https://management.azure.com/subscriptions/<azure-subscription-id>/resourceGroups/<rg>/providers/Microsoft.ApiManagement/service/<APIM_NAME>
az rest --method get --url "$APIM/subscriptions?api-version=2024-05-01&\$filter=properties/state eq 'submitted'" \
--query "value[].{name:name, scope:properties.scope, owner:properties.ownerId}" -o table
az rest --method patch --url "$APIM/subscriptions/<name>?api-version=2024-05-01" --body '{"properties":{"state":"active"}}'
The owner column is the developer account, which is how you match a request to a contract before approving it. verify.sh covers both ends: case P7 calls with the key of a subscription still in submitted and expects a 401; case P8 repeats the call after the PATCH and expects a 200. The PATCH with "state":"cancelled" from Part 1 is the same call in the other direction.
contoso-shop, the subscription Bicep created in Part 1, isn't affected by the switch to approval. It was created with state: 'active', and approvalRequired only applies to new requests. That also means the Bicep route is still open for partners you onboard without the portal: a subscriptions resource with state: 'active' skips the queue entirely.
Customising the portal, and when to self-host it
The managed portal on v2 is customisable in the visual editor: pages, layouts, styles, menus and the built-in widgets, plus a content security policy and per-group visibility for sections, menus and buttons. White-labelling stays within that: your colours and fonts in the styles, your logo as a media file, and a custom domain on the portal, whose origin then goes into the cors policy next to the default one, as the CORS section said.
What v2 limits is how much of that you can have. From the service limits for the portal on v2 tiers:
!Developer portal limits on API Management v2 tiers: pages, widgets, media files and client requests for Basic v2, Standard v2 and Premium v2
Thirty pages sounds like plenty until a partner programme wants a page per integration guide. Fifteen media files of 500 KB each is a logo, a favicon and a handful of diagrams. And three features aren't there at all: the v2 tiers overview lists Reports and the custom HTML code widget and custom widget as currently unavailable. On Basic v2 a developer can't see their own usage in the portal, and anything the built-in widgets can't render (an onboarding form posting to your CRM, an embedded status page) has no place to live.
Restricting it to your own people
For an internal API the portal should be invisible to everyone else, and that's configuration, not self-hosting (secure access):
- Require sign-in. Under the portal's site settings, Redirect anonymous users to sign-in page removes the anonymous default the portal section described.
- Sign in with Entra ID, and nothing else. Add Entra ID as the identity provider, then delete the Username and password provider, which "prevents adding users to use username and password authentication" (basic authentication). Entra ID sign-in needs an app registration of its own, which is why the sample keeps the default provider.
- Hide what's left with groups. Products visible only to an internal group, and sections of pages restricted to the same group.
What configuration can't do on v2 is take the portal off the internet. Even with VNet integration on Standard v2, "the API Management gateway, management plane, and developer portal remain publicly accessible from the internet". Sign-in is the boundary.
When self-hosting pays off
The portal's source is open, at Azure/api-management-developer-portal, and you can build and host it yourself (self-host). The self-hosted portal is a static site, typically in a storage account's static website, that still uses the managed portal's backend at https://<service name>.developer.azure-api.net for its data; "the content of all portals is stored in the same API Management service". Learn describes it as "full flexibility and extensibility": your own widgets, your own hosting and CDN, your own release cadence, the portal inside a site you already run.
It pays off when the limits above are the reason the portal doesn't do its job: you need a custom widget, or the portal has to live inside an existing site. One caveat for v2: the self-host page applies to Basic v2 and Standard v2, and v2 rules out custom widgets in the managed portal, but no Learn page says that custom widgets in a self-hosted portal are supported against a v2 instance. Check that before you plan around it.
What you pay:
- An Entra app registration. The prerequisites list "permissions to create a Microsoft Entra app registration", next to a storage account, Node.js and the Azure CLI. This is why the article doesn't show a self-hosted portal: the tenant the sample is built in doesn't allow users to create app registrations.
- The visibility controls. "Visibility and access controls are supported only in the managed developer portal. They aren't supported in the self-hosted portal." The partners group still decides who can subscribe, but no longer what a page shows.
- Upgrades. The managed portal "receives and applies software updates automatically". A self-hosted one is on the release tag you built from (not master, which is daily development) until you rebuild.
- Upkeep around it. A SAS token in the configuration that expires, notification email templates that still link to the managed portal's URL until you change them, and support that's limited: "Azure Support's assistance is limited only to the basic setup."
If the limit you hit is pages, Standard v2's 50 or a few merged pages is a cheaper fix than owning a portal build. Self-host when you need a capability the managed portal can't have; more headroom alone doesn't justify the build.
Conclusion
The order API now has a second version and a portal where new partners can find it. Partners who already call /sales/orders haven't noticed anything, because the live API joined its version set as the Original; v2 and its page envelope sit next to it at /sales/v2/orders. New developers find both in the portal, take a free key without asking anyone, and wait for a person only when they want partner limits. The decisions that are hard to undo were made once: path as the scheme, global scope for cors, groups for visibility. The ones that will change often (descriptions, limits, which developer is in partners) are a Bicep change or a click.
Those clicks are the problem Part 5 picks up. The products and version set are in Bicep, but group membership, every subscription approved by hand and any policy someone fixed in the Azure portal's editor exist only in the instance. The APIOps extractor pulls an instance's configuration out into files, and the publisher pushes those files into the next environment, so a test instance and production stop drifting apart one portal edit at a time.
Is your live API in a version set as the Original, or does it already carry a v1 identifier?
Microsoft Foundry の評価で、正答と誤答をどう採点するか試してみた
執筆日
2026/10/22
参考資料
Microsoft Foundry でのクラウド評価の概要
データセットを使ってクラウド評価を実行する
評価結果を取得する
評価に必要なアクセス許可
評価のリージョン、制限、仮想ネットワーク
azure-ai-evaluation の F1 スコア実装（GitHub）
検証内容
質問、回答、正解を手書きしたデータセットを登録し、その回答をそのまま採点しました。3件のうち1件はわざと誤答にしてあります。各エバリュエーターが正答と誤答をどう採点するか、行ごとのスコア、合否、理由を確認しました。
検証環境
モデル: gpt-6.1-sol（Global Standard）
SDK: azure-ai-projects 2.8.0, openai 3.29.0
Python: 3.13
使うエバリュエーター
名前
種類
入力
何を測るか
coherence
LLM ジャッジ（1〜5、しきい値 3）
query, response
文として筋が通っているか。事実の誤りは見ない
f1
計算式（0〜1、しきい値 0.5）
response, ground_truth
正解と単語がどれだけ重なっているか
similarity
LLM ジャッジ（1〜5、しきい値 3）
query, response, ground_truth
正解と意味が一致しているか
今回は3つですが、他にもエバリュエーターはあります。
learn.microsoft.com
事前準備
ツールのインストール
mkdir foundry-eval-handson && cd foundry-eval-handson
python -m venv .venv
source .venv/bin/activate # Windows は .venv\Scripts\activate
pip install "azure-ai-projects>=2.8.0" azure-identity openai python-dotenv
サインインと .env の作成
az login -t <tenant ID>
az account show --query "{subscription:name, tenant:tenantId}" -o table
表示されたサブスクリプションとテナントが、Foundry プロジェクトのあるものと一致することを確認してください。
プロジェクト エンドポイントは、ポータルのプロジェクト概要ページで確認できます。AZURE_AI_MODEL_DEPLOYMENT_NAME には、モデル名ではなくデプロイ名を書きます。
.env
AZURE_AI_PROJECT_ENDPOINT=https://<account_name>.services.ai.azure.com/api/projects/<project_name>
AZURE_AI_MODEL_DEPLOYMENT_NAME=gpt-6.1-sol
コード
手書きのテストデータもコードの中に持たせ、実行時に manual_data.jsonl へ書き出してからアップロードします。
evaluate_stored.py
import json
import os
import time
from datetime import datetime
from dotenv import load_dotenv
from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import TestingCriterionAzureAIEvaluator
from openai.types.eval_create_params import DataSourceConfigCustom
from openai.types.evals.create_eval_jsonl_run_data_source_param import (
CreateEvalJSONLRunDataSourceParam,
SourceFileID,
)
load_dotenv()
ENDPOINT = os.environ["AZURE_AI_PROJECT_ENDPOINT"]
MODEL = os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"] # 採点役のモデル
project_client = AIProjectClient(endpoint=ENDPOINT, credential=DefaultAzureCredential())
openai_client = project_client.get_openai_client()
実行ごとに一意なバージョンを使い、同じ name / version の再登録を避ける
RUN_VERSION = datetime.now().strftime("%Y%m%d%H%M%S")
手書きデータ。3 行目はわざと誤った回答にしている
MANUAL_DATA_PATH = "manual_data.jsonl"
MANUAL_ROWS = [
{
"query": "Contoso の返金期限は購入から何日以内ですか？",
"response": "購入から 30 日以内であれば全額返金されます。",
"ground_truth": "購入から 30 日以内で、元の状態で返品された商品は全額返金されます。",
},
{
"query": "ダウンロード済みのデジタル商品は返金できますか？",
"response": "ダウンロード後のデジタル商品は返金できません。",
"ground_truth": "デジタル商品はダウンロード後は返金できません。",
},
{
"query": "30 日を過ぎた場合はどうなりますか？",
"response": "いつでも現金で全額返金されます。",
"ground_truth": "30 日を過ぎた場合は、サポートの判断でストアクレジットが発行されることがあります。",
},
]
def build_testing_criteria() -> list:
"""エバリュエーターを定義する。response と ground_truth はデータセットの列を参照する"""
return [
TestingCriterionAzureAIEvaluator( # LLM 採点: 文として筋が通っているか（1〜5）
type="azure_ai_evaluator",
name="coherence",
evaluator_name="builtin.coherence",
initialization_parameters={"model": MODEL},
data_mapping={"query": "{{item.query}}", "response": "{{item.response}}"},
),
TestingCriterionAzureAIEvaluator( # 計算式: 正解との単語の重なり（0〜1）
type="azure_ai_evaluator",
name="f1",
evaluator_name="builtin.f1_score",
data_mapping={
"response": "{{item.response}}",
"ground_truth": "{{item.ground_truth}}",
},
),
TestingCriterionAzureAIEvaluator( # LLM 採点: 正解と意味が一致しているか（1〜5）
type="azure_ai_evaluator",
name="similarity",
evaluator_name="builtin.similarity",
initialization_parameters={"model": MODEL},
data_mapping={
"query": "{{item.query}}",
"response": "{{item.response}}",
"ground_truth": "{{item.ground_truth}}",
},
),
]
def wait_and_show(eval_object, eval_run):
"""評価実行の完了を待ち、集計と行ごとの結果を表示する"""
while True:
run = openai_client.evals.runs.retrieve(run_id=eval_run.id, eval_id=eval_object.id)
if run.status in ("completed", "failed", "canceled"):
break
print(f" status={run.status} ... waiting")
time.sleep(10)
if run.status != "completed":
raise RuntimeError(f"Evaluation run ended in {run.status}: {run.error}")
run_data = run.to_dict(warnings=False)
print("\n=== 集計 ===")
print("result_counts:", run_data.get("result_counts"))
for c in run_data.get("per_testing_criteria_results") or []:
passed, failed, errored = c.get("passed", 0), c.get("failed", 0), c.get("errored", 0)
judged = passed + failed
pass_rate = f"{passed / judged:.2f}" if judged else "-"
print(
f" {c.get('testing_criteria')}: pass_rate={pass_rate} "
f"(passed={passed}, failed={failed}, errored={errored})"
)
print("\n=== 行ごとの結果 ===")
items = list(openai_client.evals.runs.output_items.list(run_id=run.id, eval_id=eval_object.id))
for i, item in enumerate(items, 1):
item_data = item.to_dict(warnings=False)
query = (item_data.get("datasource_item") or {}).get("query")
print(f"--- item {i}: {query}")
for r in item_data.get("results") or []:
if r.get("status") == "error":
error = (r.get("sample") or {}).get("error") or {}
print(f" {r.get('name')}: ERROR {error.get('message')}")
continue
print(
f" {r.get('name')}: score={r.get('score')} "
f"label={r.get('label')} threshold={r.get('threshold')}"
)
if r.get("reason"):
print(f" reason: {r.get('reason')}")
print("\nReport URL:", run.report_url)
return run
def main():
print(f"model: {MODEL}")
print("\n##### 格納済み応答の評価 #####")
with open(MANUAL_DATA_PATH, "w", encoding="utf-8") as f:
for row in MANUAL_ROWS:
f.write(json.dumps(row, ensure_ascii=False) + "\n")
dataset = project_client.datasets.upload_file(
name="handson-manual-data",
version=RUN_VERSION,
file_path=MANUAL_DATA_PATH,
)
print(f"dataset: {dataset.name} v{dataset.version} (id: {dataset.id})")
eval_object = openai_client.evals.create(
name="handson-dataset-evaluation",
data_source_config=DataSourceConfigCustom(
type="custom",
item_schema={
"type": "object",
"properties": {
"query": {"type": "string"},
"response": {"type": "string"},
"ground_truth": {"type": "string"},
},
"required": ["query", "response", "ground_truth"],
},
),
testing_criteria=build_testing_criteria(),
)
eval_run = openai_client.evals.runs.create(
eval_id=eval_object.id,
name="manual-data-run",
data_source=CreateEvalJSONLRunDataSourceParam(
type="jsonl",
source=SourceFileID(type="file_id", id=dataset.id),
),
)
wait_and_show(eval_object, eval_run)
if name == "main":
main()
実行結果
出力（接続情報はマスク済み）model: gpt-6.1-sol
格納済み応答の評価 #####
dataset: handson-manual-data v20261021091729 (id: azureai://accounts/<account_name>/projects/<project_name>/data/handson-manual-data/versions/20261021091729)
status=in_progress ... waiting
status=in_progress ... waiting
status=in_progress ... waiting
=== 集計 ===
result_counts: {'errored': 0, 'failed': 2, 'passed': 1, 'total': 3, 'skipped': 0}
coherence: pass_rate=1.00 (passed=3, failed=0, errored=0)
f1: pass_rate=0.33 (passed=1, failed=2, errored=0)
similarity: pass_rate=0.67 (passed=2, failed=1, errored=0)
=== 行ごとの結果 ===
--- item 1: Contoso の返金期限は購入から何日以内ですか？
coherence: score=5.0 label=pass threshold=3
reason: Let's think step by step: 質問は購入から何日以内が返金期限かを尋ねており、回答は「購入から30日以内」と期限を明確に示しています。全額返金という補足も同じ条件に自然につながっており、簡潔で論理的にまとまった、理解しやすい回答です。
f1: score=0.6666666666666666 label=pass threshold=0.5
similarity: score=4.0 label=pass threshold=3
reason: 予測回答は「購入から30日以内」という返金期限と全額返金の点で正解と一致していますが、「元の状態で返品された商品」という条件が省略されています。
--- item 2: ダウンロード済みのデジタル商品は返金できますか？
coherence: score=5.0 label=pass threshold=3
reason: Let's think step by step: 質問はダウンロード済みのデジタル商品の返金可否を尋ねており、回答は返金できないと直接かつ明確に答えています。一文で完結していて論理的な混乱や不要な情報がなく、質問と回答のつながりも明瞭です。この簡潔な質問に対して必要な内容を過不足なく示しているため、非常に高い一貫性があります。
f1: score=0.0 label=fail threshold=0.5
similarity: score=5.0 label=pass threshold=3
--- item 3: 30 日を過ぎた場合はどうなりますか？
coherence: score=4.0 label=pass threshold=3
reason: Let's think step by step: The query asks what happens after 30 days, and the response directly states that a full cash refund is available at any time, implicitly including after 30 days. The sentence is clear, complete, and easy to understand, with no internal contradictions or disjointed ideas. It is coherent, though it does not elaborate on how the 30-day threshold affects the refund policy.
f1: score=0.0 label=fail threshold=0.5
similarity: score=1.0 label=fail threshold=3
reason: 正解は「30日を過ぎた場合、サポートの判断でストアクレジットが発行されることがある」としているのに対し、予測回答は「いつでも現金で全額返金される」と述べており、返金条件・返金方法・確実性がすべて異なるため、同等性はほぼありません。
Report URL: ai.azure.com<masked>/build/evaluations/<eval_id>/run/<run_id>
結果のまとめ
行
内容
coherence
f1
similarity
行の判定
1
正答
5 pass
0.67 pass
4 pass
pass
2
正答
5 pass
0 fail
5 pass
fail
3
誤答
4 pass
0 fail
1 fail
fail
スコアと合否（label）の関係
各エバリュエーターは、それぞれの尺度でスコアを出します（品質系は5段階、F1 は 0〜1 など）。そのスコアを既定のしきい値で pass / fail に変換します。coherence と similarity の既定しきい値は 3、F1 は 0.5 です。しきい値は SDK で上書きできます。
learn.microsoft.com
ポータルでの確認
評価結果の画面
実行結果に表示された Report URL をブラウザーで開く
実行の一覧から manual-data-run を開く
行ごとのスコア、合否、reason を確認する
確認すること
スクリプトの出力と、ポータル上のスコア・合否が一致しているか
同じ評価定義の中で、複数の実行を並べて比較できるか
データセットの画面
ポータルの [データ] タブを開く
handson-manual-data を開き、行をプレビューする
確認すること
登録した3行が、列名（query / response / ground_truth）どおりに表示されているか
実行ごとにバージョンが分かれて保存されているか
3 likes
Mentions
AIの回答評価にDecisionモデルが使える。速いと0.5秒、理由文は出ない RAGやエージェントの回答を、LLMに採点させて品質を見ている人向けの話です。11月3日公開の azure-ai-evaluation 1.18.11（リリースノート記載）で、GroundednessEvaluator や RelevanceEvaluator などの評価器に、採点役としてMicrosoft-Decision-1を指定できるようになりました。採点役をチャットモデルからDecisionモデルに替えると何が変わるのか、実際に動かして確かめました。 5件のうち採点された4件で返ってきたのは、点数と合否だけでした。理由文は「No model rationale is available.」を含む定型文で、判断の根拠は読めません。一方、速いときは0.5秒前後で返りました。 ! 2026-11-06、macOS・Python 3.9・azure-ai-evaluation 1.18.11で確認しました。評価した件数は5件と少なく、チャットモデル（gpt系）との採点の一致率は見ていません。確かめられたのは、設定の仕方と返ってくる値の形です。 用意するもの。Foundryリソースと、Decisionモデルのデプロイ Portalでは次の2つの画面を確認しました。まず、Foundry（AIServices）のリソースを japaneast に作ります。 Azure Portalで対象リソースを開いた概要です。「Foundryポータルに移動」から次の画面へ進めます。 次にFoundryポータルの「モデル + エンドポイント」で、Microsoft-Decision-1をデプロイします。名前は dec1 にしました。 左メニューの「モデル + エンドポイント」を開くと、デプロイ名と状態が並びます。 デプロイ方式は DataZoneStandard、容量は1にしました。リソース作成とデプロイを az で行う場合は次のとおりです。 az で作る場合（コマンド）az group create -n <RG名> -l japaneast az cognitiveservices account create -n <リソース名> -g <RG名> -l japaneast \ --kind AIServices --sku S0 --custom-domain <リソース名> --yes az cognitiveservices account deployment create -n <リソース名> -g <RG名> \ --deployment-name dec1 --model-name Microsoft-Decision-1 --model-version 1 \ --model-format Microsoft --sku-name DataZoneStandard --sku-capacity 1 評価器に渡す設定は4つだけ 評価器のモデル設定に type="decision" を付けます。エンドポイントはプロジェクトやOpenAI用のURLではなく、リソースのルートを渡します。 from azure.ai.evaluation import GroundednessEvaluator cfg = { "type": "decision", "azure_endpoint": "https://<リソース名>.cognitiveservices.azure.com/", "azure_deployment": "dec1", "api_key": "<キー>", } ev = GroundednessEvaluator(cfg) r = ev(query="…", response="…", context="…") 認証はAPIキーか、自分で渡した credential です。ソースを読むと、Decision用の既定の認証情報は作られません。DefaultAzureCredential を使いたいときは、自分で作って credential に渡します。 設定に決まっていない項目を足すとエラーになります。max_request_bytes を足して試したところ、Unsupported decision model configuration fields で止まりました。使える項目は type・azure_endpoint・azure_deployment・api_key・credential・probability_threshold です。 結果。点数と合否は出る。理由文は定型文 groundedness（回答が渡した文脈に基づいているか）と relevance（質問に答えているか）を、各2〜3件ずつ採点させました。文脈は「セマンティックランカーはBasic以上のSKUで使える。Freeでは使えない」の1文です。 結果を表に整形した画像です。groundednessとrelevanceの閾値はどちらも3でした。 ケース 回答の内容 スコア 結果 g_ok 文脈どおり「Basic以上」 5 pass g_bad 文脈に反して「Freeでも使える」 2 fail g_nocontext 文脈なしで「東京です」 なし not_applicable r_ok 質問に答えている 4 pass r_off 桜の話で質問と無関係 1 fail 文脈と食い違う回答は2点で落ち、質問と無関係な回答は1点で落ちました。文脈を渡さない通常の文章は、groundednessが採点せず not_applicable で返りました。リリースノートの説明と同じ挙動でした。 結果で気づいたことは次の3つです。 理由文は定型です。 groundedness_reason は「Selected groundedness level 2 from the highest-probability level. No model rationale is available.」でした。なぜ2点なのかは文章では分かりません。今回はチャットモデルの評価器を動かしていないため、理由文の有無の比較はしていません。 groundednessの1回の採点で入力は約340トークン、出力は2トークンでした。出力はごく小さいです。 所要時間はばらつきました。 groundednessの2件は0.4〜0.5秒でした。relevanceの2件は attempt_count が2で、23秒と48秒かかりました。再試行が1回入っています。原因は記録されておらず、最終的なHTTPステータスは200でした。評価の間は12秒ずつあけていましたが、2件だけの結果なので、再試行が常に入るのかは分かりません。 結果の properties には、使ったデプロイ名・ルーブリックのハッシュ・試行回数なども入ります。後から「どの採点基準で出た点か」を追えます。 明日試すなら 手元の評価コードがチャットモデルで動いているなら、モデル設定を差し替えるだけで試せます。まず1〜2件で次の2点を見てください。 同じ入力でチャットモデルとDecisionモデルの点数が近いか 理由文が無くても、運用上の確認で困らないか 理由文が必要な場面では、先に自分の環境でチャットモデルの評価器の出力を確かめてください。 参考 azure-ai-evaluation 1.18.11 リリースノート
AI Risk Management Training for Executives: Put a Number on It Ask a security leadership team to rank "prompt injection in the corporate AI assistant" against "ransomware on the file servers" for next year's budget, and most will answer with a color. Amber, probably. Then the CFO asks how much amber costs, and the conversation stalls. That stall is the gap that AI risk management training for security executives should close. Most courses spend the day on frameworks (NIST AI RMF, ISO/IEC 42001, the EU AI Act) and send leaders home able to classify a risk but unable to price it. Classification decides whether something goes on the register. Price decides whether it gets money. If a course does not teach the second skill, the register entry sits in amber forever. Why the heat map fails for AI risk The problem with risk matrices is not new. Louis Anthony Cox Jr. laid it out in "What's Wrong with Risk Matrices?" (Risk Analysis, 2008): coarse likelihood and impact bins can assign identical ratings to risks that differ by orders of magnitude, and in some cases rank a smaller risk above a larger one. Douglas Hubbard and Richard Seiersen made the security-specific version of that argument in How to Measure Anything in Cybersecurity Risk. AI makes it worse. "Likelihood: medium" for prompt injection hides the one number you can actually measure, which is how often an attack succeeds against your deployment. Throwing that measurement away to pick a bin is a bad trade. Decompose one scenario with FAIR The Open Group's Open FAIR standards (O-RT for the taxonomy, O-RA for the analysis) split risk into loss event frequency and loss magnitude. Loss event frequency is threat event frequency times susceptibility: how often an attacker tries, and what fraction of tries succeed. Take one concrete scenario. An external sender plants instructions in an email or shared document, your assistant retrieves it while answering an unrelated question, and the model leaks internal data through a rendered link. MITRE ATLAS calls the technique [AML.T0051.001, indirect prompt injection](/atlas/AML.T0051.001), and the outcome [AML.T0057, LLM Data Leakage](/atlas/AML.T0057). EchoLeak (CVE-2025-32711) in Microsoft 365 Copilot followed this exact path; our [AI incident tabletop post](/blog/executive-ai-risk-training-tabletop) walks through it as an exercise. Three inputs, and they come from three different places: - Attempt frequency (estimated). How many crafted payloads reach the retrieval index per year. You will not know this. Give a calibrated 90 percent range instead, say 2 to 20, and write down why. - Susceptibility (measured). The fraction of attempts that succeed. This is where AI risk differs from most FAIR work: your red team can run the attack 60 times against your own tenant with PyRIT or garak and count. Seven successes in 60 trials is evidence. "The vendor says it is mitigated" is not. - Loss per event (estimated with finance and legal). Response hours, notification, counsel, regulatory exposure. For a sense of the upper end, IBM's Cost of a Data Breach Report 2025 puts the global average breach at US$4.44 million, and GDPR Article 83 allows fines up to 4 percent of worldwide annual turnover. A range of $50,000 to $2 million per leak event is a defensible starting point for most mid-sized firms; yours may differ. The simulation About twenty lines of NumPy turn those inputs into a loss distribution. Ranges become lognormals, red team counts become a Beta distribution, and attempts become Poisson draws. import numpy as np rng = np.random.default_rng(7) N = 100_000 def lognormal_from_ci(low, high, size): # treat (low, high) as a calibrated 90% interval mu = (np.log(low) + np.log(high)) / 2 sigma = (np.log(high) - np.log(low)) / (2 * 1.645) return rng.lognormal(mu, sigma, size) def simulate(successes, trials): attempts = rng.poisson(lognormal_from_ci(2, 20, N)) # payloads reaching retrieval per year p_success = rng.beta(1 + successes, 1 + trials - successes, N) # red team result events = rng.binomial(attempts, p_success) # loss events per year return np.array([lognormal_from_ci(50_000, 2_000_000, k).sum() for k in events]) before = simulate(successes=7, trials=60) # current config after = simulate(successes=0, trials=60) # external mail excluded from retrieval, link allowlist on for name, ale in [("before", before), ("after", after)]: print(f"{name}: mean ${ale.mean():,.0f} P90 ${np.percentile(ale, 90):,.0f} " f"P(loss > $1M) {np.mean(ale > 1e6):.1%}") You can sanity-check the means without running anything. The 2 to 20 attempt range has a mean of about 8. The Beta posterior for 7 of 60 has a mean of 8/62, about 13 percent. The $50,000 to $2 million loss range has a mean near $590,000. Multiply them: roughly $620,000 a year in expected loss before the fix. After the fix, the success rate drops to a posterior mean of 1/62, and expected loss falls to about $80,000. That difference, around $540,000 a year, is the number to put next to the cost of the control. It is also the number that tells you the control does not get you to zero. Zero successes in 60 trials still leaves a 95 percent upper bound of about 5 percent (the rule of three: 3/n), and the simulation carries that uncertainty instead of rounding it away. The P90 and P(loss > $1M) lines matter more than the mean for a board. "One year in ten, this scenario costs us more than X" is a sentence directors can act on. NIST's AI RMF asks for this work without naming a method: MAP 5.1 covers documenting likelihood and magnitude of impacts, and MEASURE 2.7 covers evaluating security and resilience. A script an auditor can rerun meets both. If you want a packaged version, pyfair implements the FAIR ontology in Python. What a course should make you do Judge any AI risk management course by one exercise: does each participant leave having priced a scenario from their own environment, with at least one measured input? Lectures on likelihood scales do not count. Neither does a vendor's risk calculator with the inputs pre-filled. The measured input is the part executives tend to skip, because it requires asking the red team for trial counts instead of a pass/fail verdict. Our post on [counting trials in AI red team work](/blog/ai-red-team-training-security-engineers) explains what to request. Calibration is the other piece: Hubbard's equivalent-bet test (would you rather bet on your range containing the answer, or on a spin with a 90 percent chance of winning?) takes twenty minutes to teach and fixes most overconfident ranges. Where quantification breaks It needs an inventory first. You cannot price a scenario for an assistant nobody knew was connected to the mail server; our [CISO post](/blog/where-cisos-learn-ai-security) covers finding those through proxy logs. It also decays. A measured success rate describes one model version under one system prompt, and SaaS vendors swap models without notice, so the red team rerun belongs on a schedule (quarterly, and after any vendor model change). And it handles the tail poorly. A regulator deciding to make an example of the first AI leak in your sector is not well described by a lognormal; say that out loud rather than letting the P90 imply more precision than it has. None of this replaces judgment. It gives judgment something to argue with. GTK Cyber's [A Cyber Executive's Guide for Artificial Intelligence](/courses/executive-ai-guide) covers AI risk and governance frameworks and ROI evaluation for AI security tools, and we deliver it privately for leadership teams through an [executive briefing](/executive-briefing).
Azurite - A lightweight server clone of Azure Storage that simulates most of the commands supported by it with minimal dependencies Azure/Azurite
Azure API Management Developer Portal and API Versioning: Version Sets Without Breaking Callers How do you ship a breaking change to the order API without breaking the partners who already call it, when the next partner should find the API in the developer portal, read the docs and get a key without opening a ticket? The versioning half has a trap that decides the rest: API Management fixes the versioning scheme for every version in a version set the moment you add the first one, and the live API keeps its URL only if it stays the Original version. Give it a v1 identifier and every existing call stops matching. The short answer for the order API: path versioning, the live API kept as Original, v2 only for the breaking change, revisions for everything else, and two products so a free tier and partners get different limits and only partners need approval. After Part 3 the order API is locked down and nobody can find it. There is one product, one subscription that Bicep created for a partner called Contoso Shop, and an Entra ID token check in front of every operation. A new partner learns the API exists from an email, gets the OpenAPI file as an attachment and waits for someone to create a key. The companion repo now enables the developer portal, adds a global CORS policy so the portal's test console can reach the gateway, puts the existing API into a path-based version set next to a v2 with a paged list-orders, and splits access into an auto-approved free product and a partner product that needs approval. The instance is still Basic v2, and that has two consequences: the portal is off until you turn it on, and the managed portal can't host custom widgets. The Bicep and the function project build clean; none of it has been deployed yet, so where the article depends on how a live instance behaves, it says so and points to the check in the repo's scripts/verify.sh. What the developer portal builds from your OpenAPI file The portal's API reference is the OpenAPI file you imported in Part 1, rendered. This is list-orders in openapi/orders.yaml: paths: /orders: get: operationId: list-orders summary: List orders description: Returns all orders, newest first. Filter with `status`. parameters: - name: status in: query required: false description: Only orders in this status. schema: $ref: '#/components/schemas/OrderStatus' responses: '200': description: Orders, newest first. content: application/json: schema: type: array items: $ref: '#/components/schemas/Order' Learn calls the portal "an automatically generated, fully customizable website with the documentation of your APIs", and "out of the box, the developer portal is already populated with your published APIs and products" (overview). For the order API that means a list of APIs, a reference page per operation built from the imported definition (method, URL, parameters, request and response schemas), a products page, sign-up and sign-in, and a profile page where a signed-in developer sees their subscription keys. Nobody writes these pages; they change when the API definition changes. Three defaults decide what a visitor gets: - Only published products show up. A product's state: 'published' makes it discoverable in the portal, and that's all it does. Learn is explicit that the gateway doesn't care: "Regardless of whether the product is published, all of the product's APIs are accessible for requests via the API Management gateway" (products). Unpublishing a product hides it from new developers; it doesn't revoke a single key. - The profile page only lists product keys. "Currently, the developer portal only shows the product scope subscriptions under the User Profile section" (subscriptions). A subscription scoped to one API or to all APIs works at the gateway and is invisible to its owner in the portal. If developers are meant to manage their own keys, scope every subscription to a product. - Anyone can read it. "By default, the developer portal enables anonymous access." Anonymous visitors are in the built-in Guests group and see the pages and the products made visible to Guests, until you switch on the site setting that requires sign-in. For many partner APIs that's fine, since the docs aren't the secret; the keys are. If the API's shape is itself confidential, keep its products away from Guests and require sign-in before the first publish, not after. Sign-up uses API Management's own user accounts by default; Entra ID and Entra External ID sign-in are the alternatives (not on the classic Basic tier or Consumption). A user who signs up lands in the built-in Developers group, and which products a developer can see is decided per group. The products section below uses that to keep the partner product out of sight for everyone outside a partners group. Turning it on in Basic v2 On the classic tiers the portal exists from the moment the instance does. On v2 tiers it doesn't: "If you created your instance in a v2 service tier, first enable the developer portal" (customize). In the Azure portal that's Developer portal > Portal settings > Enabled > Save, which "might take a few minutes". In Bicep it's one property on the service in infra/apim.bicep: properties: { publisherEmail: publisherEmail publisherName: 'Orders demo' // v2 tiers create the instance with the portal off; Consumption has no portal at all. developerPortalStatus: apimSku == 'Consumption' ? null : 'Enabled' } developerPortalStatus takes 'Enabled' or 'Disabled' in resource API version 2024-05-01, which the sample already uses (template reference). The sample's apimSku parameter still allows Consumption, and Consumption is the one tier without a managed portal, so the property is null there instead of asking for something the tier can't do. Learn says the enable step works on every v2 tier. One Microsoft Q&A answer (not documentation) reports an instance where the option was missing on Basic v2. This sample hasn't been deployed with the property yet, so treat Learn as the authority and check your own instance: after the deployment, Portal overview in the Azure portal should show the portal URL, https://<service name>.developer.azure-api.net. Enabling the portal doesn't make it visible. The portal content has to be published, and the first publish is a manual step: open Portal overview, select Developer portal to open it in the administrative interface, then Publish. The same rule applies to every later change, because "Saved changes are visible only to you and aren't visible to portal visitors until you publish them." Learn also lists service changes that need a republish: a custom domain, identity providers, delegation, sign-in and product terms. Changes to an API definition are not on that list. That fits Learn's split between portal content (pages, styles, layouts) and API Management data (APIs, operations, products), but it's an inference; republish once after the first import and check. No Learn page shows publishing from Bicep, so the sample leaves it as a README step. Descriptions and examples that end up on the page Learn doesn't enumerate which OpenAPI fields the reference page renders, so the sample fills in the ones a partner would look for, and you check the result on your instance before relying on any of them. The order spec now carries three kinds of text: - an operation description that says what the call returns and how to filter it, - a description on each parameter, which is where a partner looks to find out what status filters on, - example values on schema properties, from components.schemas in the same file: required: [sku, quantity, unitPrice] properties: sku: type: string example: BOOK-0042 quantity: type: integer minimum: 1 example: 2 unitPrice: type: number format: decimal example: 24.90 Write these for the person reading the portal, not for the code generator. Only orders in this status. is short, but it answers the question a partner has at that spot; a description that repeats the parameter name in a sentence answers nothing. The examples matter more than they look: a developer copies them into their first request, so keep the set consistent (the same BOOK-0042 everywhere). Two more texts reach the portal without touching the spec. When you make a revision current, the optional change log "is included in the developer portal for your API users to view" (revisions), which is the only release-notes channel the portal has built in. And each API version carries its own spec, so the v2 reference page documents the page envelope while the Original keeps documenting the array; the versioning section below covers how the two coexist. The try-it console and the CORS error everyone hits Every operation page has a Try it button. It opens a console that fills in the request URL, takes a subscription key (a signed-in developer can pick one of their own), lets you set parameters and headers, and sends the call. Learn lists three authorization models for it: no authorization, a subscription key, and OAuth 2.0, where "you can configure the test console to generate a valid OAuth token on behalf of the test console user". The first time someone selects Send against a fresh instance, the browser shows a CORS error and nothing reaches the API. The console is JavaScript running on https://<service name>.developer.azure-api.net and calling https://<service name>.azure-api.net, a different origin. The subscription key travels in the Ocp-Apim-Subscription-Key header, a custom header, so the browser first sends an OPTIONS preflight asking whether the portal's origin may send it. Without a cors policy the gateway's answer carries no Access-Control-Allow-Origin, and the browser stops there. Learn: "If the developer portal's domain name isn't an allowed origin for cross-domain API requests, test console users will see a CORS error" (enable CORS). The fix is a cors policy at the top of the global policy, infra/policies/global.xml: <inbound> <cors allow-credentials="false"> <allowed-origins> <origin>{{portal-origin}}</origin> </allowed-origins> <allowed-methods preflight-result-max-age="300"> <method>*</method> </allowed-methods> <allowed-headers> <header>*</header> </allowed-headers> <expose-headers> <header>*</header> </expose-headers> </cors> The origin comes from a named value, built in apim.bicep from the instance name and added to the named values the policies already read: // The default portal host; add a custom portal domain here too, or the try-it console gets a CORS error there. var portalOrigin = 'https://${apim.name}.developer.azure-api.net' // Plain (non-secret) values the policies read as {{name}}: IDs and the portal origin, not credentials. var policyNamedValues = { 'tenant-id': tenant().tenantId 'orders-api-app-id': ordersApiAppId 'orders-backend-app-id': backendAppId 'orders-test-client-app-id': testClientAppId 'portal-origin': portalOrigin } What each part of the policy decides: - One origin, not *. The cors policy reference wants a full URI with scheme and host for each origin, and warns that * "may be overly permissive". Allowing every origin would let any website call your API from a visitor's browser with whatever key that page holds. With a custom domain on the portal, add its origin as a second <origin>; the custom domain setup can add it for you. - allow-credentials="false". The console sends the key and, with OAuth, the token as headers. It has no cookies to send to the gateway, so there is no reason to allow them. This is the policy's default; it's spelled out so nobody flips it to true while debugging. - Methods and headers *. The console sends whatever the operation needs: POST with Content-Type for create-order, the key header, Authorization. Listing them one by one is tighter, and it's also the list you forget to update when the next API arrives on the same instance. - At the top of inbound, before everything else. Learn's troubleshooting list warns of "unexpected behavior if the cors policy is not the first policy in the inbound section". It also documents why the Part 3 token check doesn't break the preflight: "Only the cors policy is evaluated on the OPTIONS request during preflight. Remaining configured policies are evaluated on the approved request." The OPTIONS call carries no key and no token, and it doesn't need to. Global, not per API, and only once. The portal's own Enable CORS button on Portal overview writes the same kind of policy at the global scope, for all APIs, and that's the right place for it: the v2 API and every later API inherit it. Take the warning on Learn's portal page literally: "Only one CORS policy is executed. If you specify multiple CORS policies (for example, on the API level and on the all-APIs level), your interactive console may not work as expected." The cors reference says you may configure it at more than one scope, which reads like a contradiction until you note that "generally, only the first cors policy is applied". Keep one, globally, and you never have to work out which one won. Never on the product. Putting the policy on orders-partners looks tidy, since only partners should use the console. It doesn't work: "If you apply the CORS policy in the Product scope, instead of the API(s) scope, and your API uses subscription key authentication through a header, your console won't work." The preflight carries no subscription key, so API Management can't tell which product the call belongs to, and a product-scoped policy never runs for it. Learn's workaround is the key in the query string, which Part 1 argued against: the key ends up in every access log between the browser and the gateway. Move the policy to global scope instead. When the gateway itself isn't reachable from the browser (a network-isolated instance), the console can route its calls through the portal's backend instead: the Operation: Details widget has a Use CORS proxy setting, and after saving you republish the portal. That needs no cors policy at all, at the price of the portal backend making the calls for you. scripts/verify.sh sends both preflights after a deployment (cases P4 and P5): one from the portal origin, which should come back 200 with Access-Control-Allow-Origin, and one from https://example.com, which should get an empty 200 without that header, the policy's documented behaviour for an origin it doesn't match. The Entra token the console doesn't have With CORS fixed, the order API's console call still fails, now with a 401. Part 3 put validate-azure-ad-token on the API, and the console sends a subscription key and nothing else. That's the same call as case V2 in the verify script: key, no token, rejected at the gateway. The v2 API from the versioning section reuses the same API policy file, on purpose: a new version shouldn't be a way around the token check. The console can get a token, through OAuth 2.0 user authorization (Learn). You register a second Entra app that represents the console, add an OAuth 2.0 authorization server to API Management pointing at Entra ID, and select that server in the API's settings. The console then shows an authorization drop-down, signs the developer in with the authorization code flow and adds the Authorization header to the call. As Part 3 quoted from the same page, this "only enables the developer portal's test console (and the test console in the Azure portal) as a client to acquire a token"; validating the token stays the policy's job. Even with OAuth set up, Part 3's policy would still reject the console's token, because it was written for partner backends that call as themselves. Two checks get in the way. Part 3's policy pins the calling app in client-application-ids to the test client, and the console's token is issued to the console's own app, so its azp is the wrong app ID. The policy also requires Orders.ReadWrite in roles, and that role is defined with allowedMemberTypes: ['Application'], so no signed-in developer can be assigned it. The console keeps getting a 401 until the policy accepts the console app's ID and the role (or a separate one) allows user assignments. That follows from the claims Part 3 walked through, not from a Learn page about the console, and it hasn't been tested here. This article doesn't show the console calling the order API. Every step above needs at least one more app registration, and the tenant the sample is built in doesn't allow users to create them, the same constraint that kept Part 3's token checks a runbook. With a subscription key alone, the console works for any API that doesn't check a token. For the order API, the options are OAuth configured as above, or reference pages whose console can't complete a call to it. Versioning: one scheme per version set, chosen once "Turning on versioning" in API Management means creating a version set, and a version set holds very little: a display name and a scheme. The versions themselves are ordinary APIs. "Each version of an API is maintained as its own API resource and is associated with a version set" (versions), so v2 of the order API isn't a flag on the existing API. It's a second API resource with its own spec, operations and policies that shares a path and a portal entry with the first. The scheme is the part you can't take back. "All versions in a version set have the same versioning scheme. It's based on the versioning scheme that's used when you first add a version to an API." There is no migration from header to path later. A different scheme means a new version set, and a new version set means every caller changes how they ask for a version. Path: the version in every log line The sample's version set in infra/apim.bicep. ARM calls path versioning Segment: // Path versioning (Segment in ARM), fixed for every version in the set by the first one. resource ordersVersionSet 'Microsoft.ApiManagement/service/apiVersionSets@2024-05-01' = { parent: apim name: 'orders' properties: { displayName: 'Orders API' versioningScheme: 'Segment' } } The existing API from Part 1 joins it with one new property: // The Original version: in the set, but no apiVersion, so existing callers keep /sales/orders. // Giving it apiVersion: 'v1' would move it to /sales/v1/orders and break every existing call. resource ordersApi 'Microsoft.ApiManagement/service/apis@2024-05-01' = { parent: apim name: 'orders' properties: { displayName: 'Orders API' apiVersionSetId: ordersVersionSet.id path: 'sales' protocols: ['https'] format: 'openapi' value: loadTextContent('../openapi/orders.yaml') serviceUrl: functionBaseUrl subscriptionRequired: true } } There's deliberately no apiVersion line. Without it, the API is the Original: "If you add a version to a non-versioned API, an Original version will be automatically created and will respond on the default URL, without a version identifier specified. The Original version ensures that any existing callers aren't affected." /sales/orders keeps answering, and every partner integration keeps working without a release on their side. Adding apiVersion: 'v1' next to v2 looks tidier, and it's the trap from the introduction. An API with an identifier answers only on its versioned URL, so the live API moves to /sales/v1/orders. The deployment is green, and every existing call to /sales/orders now matches no API at all. Learn doesn't document the status code for a call that matches no version, so verify.sh case P3 calls /sales/v9/orders and prints whatever comes back rather than asserting a number. Learn describes the Original for the Azure portal's flow, where the existing API is linked to the set "but a version identifier doesn't need to be specified". It doesn't show the ARM form. Leaving apiVersion out is the direct translation, and the Bicep builds, but whether a deployed instance keeps the Original on its old URL is what case P1 checks, and P1 hasn't run. If ARM insists on an identifier, the README's fallback is v1, with the URL change announced to partners before the deploy rather than discovered by them after it. v2 is the second API resource: // v2 breaks one contract: list-orders returns a page envelope. Same path, identifier in the URL: /sales/v2/orders. resource ordersApiV2 'Microsoft.ApiManagement/service/apis@2024-05-01' = { parent: apim name: 'orders-v2' properties: { displayName: 'Orders API' apiVersionSetId: ordersVersionSet.id apiVersion: 'v2' apiVersionDescription: 'list-orders returns a page of orders ({ items, nextLink }) instead of an array.' path: 'sales' protocols: ['https'] format: 'openapi' value: loadTextContent('../openapi/orders-v2.yaml') serviceUrl: functionBaseUrl subscriptionRequired: true } // Two APIs on one path only work inside a version set, so the Original has to join it first. dependsOn: [ordersApi] } Same path, same version set, identifier v2: the gateway serves it at /sales/v2/orders. The dependsOn exists because two APIs may share sales only as members of one version set. ARM would otherwise be free to create orders-v2 while the Original is still a standalone API on that path. The breaking change lives in openapi/orders-v2.yaml: list-orders returns { "items": [...], "nextLink": "orders?skip=50" } instead of a bare array. Every client that deserialises the response into a list breaks on that, which is what makes it a version and not a revision. The function app gets one new function, list-orders-v2 on route v2/orders, and the v2 operation gets a policy that sends it there: <inbound> <base /> <rewrite-uri template="/v2/orders" copy-unmatched-params="true" /> </inbound> copy-unmatched-params carries status and skip over to the rewritten URL. Everything else in v2 (the other two operations, the API policy, the token check) is the same as in the Original. Path versioning costs you clean resource URLs: /sales/v2/orders and /sales/orders are the same orders. In return the version is in every access log line, every bug report that pastes a URL, and every curl a partner sends to support. For server-to-server partners on mixed stacks, I'd take that visibility over clean URLs. Header: clean URLs, invisible version The sample doesn't use it, but the version set would differ in two properties (apiVersionSets reference): properties: { displayName: 'Orders API' versioningScheme: 'Header' versionHeaderName: 'Api-Version' } Every caller now sends Api-Version: v2, and the URL stays /sales/orders for all versions. The cost moves to the client side, where you can't see it: - A forgotten header isn't an error. A request without the header is a request without an identifier, and with an Original in the set that's the Original's URL. A partner who upgrades their client code to the v2 response shape but misses the header gets the old array back, and the bug surfaces as a deserialisation failure in their code, not at your gateway. - The version disappears from logs. Access logs and pasted URLs no longer say which version was called. You get it back only by logging the header explicitly. - Every hop has to keep the header. A curl needs -H, a browser address bar can't send it at all, and any proxy or HTTP cache between partner and gateway has to pass the header through and key on it. A cache that keys on the URL alone needs Vary: Api-Version, or it serves one version's response to another version's caller. That's general HTTP caching behaviour, not something API Management documents. The try-it console needs one more thing. The version header is a custom header, so it triggers a preflight, and the cors policy has to allow it. The sample's global policy allows *, which covers it; if you tighten allowed-headers to a list, add the version header to it. That follows from how CORS works and isn't stated on Learn. What Learn does state is the scope rule: "If you configure the cors policy at the API scope, and your API uses a header-versioning scheme, the policy won't work because the version is passed in a header" (cors policy). The preflight doesn't carry the version header's value, so the gateway can't pick the version whose policy should answer. It's the product-scope problem again, with a header in place of the key, and the global policy from the CORS section avoids both. Query string: the parameter that drops out of your spec versioningScheme: 'Query' with versionQueryName: 'api-version' gives /sales/orders?api-version=v2. It's Azure's own convention (the az rest calls later in this article send ?api-version=2024-05-01 to ARM), it shows up in logs, and it works from an address bar. This one bites in the tooling rather than at the gateway. Learn: "Query parameters aren't allowed in the servers property of an OpenAPI specification. If you export an OpenAPI specification from an API version, a query string won't appear in the server URL." A partner who downloads the v2 spec from the portal and generates a client from it gets a client that never sends api-version, which means it calls the Original and receives the v1 contract. The same goes for anything else that treats the exported file as the contract, such as a contract test or a mock server. If you pick query versioning, document the parameter in each operation of the spec yourself, and don't assume an exported definition round-trips. Versions or revisions Most changes to the order API shouldn't be a version at all. Learn's rule of thumb: "versions are used to separate API versions that have breaking changes, and revisions can be used for minor and non-breaking changes" (revisions). A new optional field, a new operation, a policy fix: those are revisions. A revision is a numbered copy of the same API that you edit and test without callers noticing, at https://<service name>.azure-api.net/sales;rev=2/orders (the ;rev=n goes after the API path, before the operation). Making it current is the release, and the optional change log from the portal section is the release note. The mistake to watch for is the reverse: a breaking change shipped as a revision. "It's only a field rename" goes out as rev=3, and every partner gets it at the moment someone selects Make current, with no URL to opt out on. If a revision turns out to be breaking while it's still being tested, the Azure portal can turn it into a version with Create Version from this Revision, which keeps the work and puts the change behind its own identifier. Some properties can't change in a non-current revision at all (name, path, protocols, subscription required, API version among them), so path and scheme changes are version-level decisions anyway. The four options side by side: !Path, header and query-string versioning compared with a revision: what the caller sends, log visibility, OpenAPI export, CORS, caching and the main risk Picking by situation: - External partners, many client stacks, support by email: path, with the live API as the Original. That's the order API's choice. - Clients you build and ship yourself, URLs stored where you can't easily change them: header, and keep an Original so a missing header has a defined answer. - An API that already follows the Azure-style api-version convention: query string, with the parameter written into the spec by hand. - Anything a correct client wouldn't notice: a revision, whichever scheme the set uses. Products as subscription tiers Part 1 put the order API into one product and Part 2 gave that product a rate limit. A portal with self-service sign-up needs a second one, because the developer who signs up on a Sunday to try the API shouldn't get the same key a contracted partner gets. Both products contain the same APIs; the differences are approval, the subscription limit and the policy. The free tier in infra/apim.bicep: // Evaluation: auto-approved, one subscription per user account, low limits. resource freeProduct 'Microsoft.ApiManagement/service/products@2024-05-01' = { parent: apim name: 'orders-free' properties: { displayName: 'Orders (free)' description: 'Try the order API: 5 calls a minute, 1,000 a week. Keys are issued right away.' subscriptionRequired: true approvalRequired: false subscriptionsLimit: 1 state: 'published' } } The partner product is the Part 1 resource with approvalRequired switched to true. Their policies, product-orders-free.xml and then product-orders-partners.xml, differ only in the inbound section: <inbound> <base /> <rate-limit calls="5" renewal-period="60" /> <quota calls="1000" renewal-period="604800" /> </inbound> <inbound> <base /> <rate-limit calls="60" renewal-period="60" /> </inbound> The free tier has two limits because they stop different things. rate-limit caps bursts: the sixth call inside a minute gets a 429 (verify.sh case P6). quota caps volume: 1,000 calls in 604,800 seconds, a week, after which "the caller receives a 403 Forbidden response status code" with a Retry-After header (quota). Five calls a minute without a quota still allows about 50,000 calls a week from a key nobody reviewed. Two details of quota matter here: it's allowed only at product scope, and the week starts "relative to the start time of the subscription", so every free key has its own week and no Monday-morning reset. The partner product's 60 calls a minute raises Part 2's 10. As Part 2 showed, product policies only run for product-scoped keys; the built-in all-access key the Azure portal's test tab uses skips both limits. subscriptionsLimit: 1 means one free subscription per user account. It isn't one per person: nothing stops someone from signing up with five email addresses and calling with five keys at 25 calls a minute. The free tier's numbers are set low enough that this buys little, and the partner product, where the real throughput is, sits behind approval. The 2024-05-01 products template reference also says the property "can be present only if subscriptionRequired property is present and has a value of false", which contradicts the portal tutorial and the sample's protected product. The same reference for older API versions such as 2019-12-01 says "a value of true", so the reference versions disagree; the Bicep builds, and it's unverified until a deployment accepts it. Since each version is its own API, each version goes into each product: // Every version is its own API, so each one goes into each product. var orderApiVersions = [ordersApi.name, ordersApiV2.name] resource partnerProductApis 'Microsoft.ApiManagement/service/products/apis@2024-05-01' = [for api in orderApiVersions: { parent: partnerProduct name: api }] resource freeProductApis 'Microsoft.ApiManagement/service/products/apis@2024-05-01' = [for api in orderApiVersions: { parent: freeProduct name: api }] Forget this for v2 and it's in no product: "To publish the API, you must associate it with a product" (publish versions). No product key can call it, and the portal has no product to show it in. Who sees which product is decided by groups: "Products are first made visible to groups, and then developers in those groups can view and subscribe to the products that are associated with the groups" (groups). The sample makes the free product visible to the built-in developers group (everyone who signed in) and the partner product visible to a custom partners group, both through products/groups resources. Neither is visible to Guests, the group anonymous visitors fall into. Approving a partner subscription A partner's path through the portal, with the sample's settings: 1. They sign up and land in Developers. They see the free product and can take a key straight away. 2. Someone on the API team adds their account to partners (Azure portal > the instance > Users). Only now does Orders for partners appear for them. 3. They request it. Because the product requires approval, the new subscription is created in the submitted state, which the subscriptions template reference defines as "request has been made by the developer, but has not yet been approved or rejected". 4. API Management sends the Subscription requests (requiring approval) notification to whoever is configured for it under Notifications on the instance (notifications). If nobody is configured, nobody gets an email and the request just sits there. 5. Someone approves it, in the Azure portal under Subscriptions or from a script. The script version, from the README, lists the waiting subscriptions and activates one: APIM=https://management.azure.com/subscriptions/<azure-subscription-id>/resourceGroups/<rg>/providers/Microsoft.ApiManagement/service/<APIM_NAME> az rest --method get --url "$APIM/subscriptions?api-version=2024-05-01&\$filter=properties/state eq 'submitted'" \ --query "value[].{name:name, scope:properties.scope, owner:properties.ownerId}" -o table az rest --method patch --url "$APIM/subscriptions/<name>?api-version=2024-05-01" --body '{"properties":{"state":"active"}}' The owner column is the developer account, which is how you match a request to a contract before approving it. verify.sh covers both ends: case P7 calls with the key of a subscription still in submitted and expects a 401; case P8 repeats the call after the PATCH and expects a 200. The PATCH with "state":"cancelled" from Part 1 is the same call in the other direction. contoso-shop, the subscription Bicep created in Part 1, isn't affected by the switch to approval. It was created with state: 'active', and approvalRequired only applies to new requests. That also means the Bicep route is still open for partners you onboard without the portal: a subscriptions resource with state: 'active' skips the queue entirely. Customising the portal, and when to self-host it The managed portal on v2 is customisable in the visual editor: pages, layouts, styles, menus and the built-in widgets, plus a content security policy and per-group visibility for sections, menus and buttons. White-labelling stays within that: your colours and fonts in the styles, your logo as a media file, and a custom domain on the portal, whose origin then goes into the cors policy next to the default one, as the CORS section said. What v2 limits is how much of that you can have. From the service limits for the portal on v2 tiers: !Developer portal limits on API Management v2 tiers: pages, widgets, media files and client requests for Basic v2, Standard v2 and Premium v2 Thirty pages sounds like plenty until a partner programme wants a page per integration guide. Fifteen media files of 500 KB each is a logo, a favicon and a handful of diagrams. And three features aren't there at all: the v2 tiers overview lists Reports and the custom HTML code widget and custom widget as currently unavailable. On Basic v2 a developer can't see their own usage in the portal, and anything the built-in widgets can't render (an onboarding form posting to your CRM, an embedded status page) has no place to live. Restricting it to your own people For an internal API the portal should be invisible to everyone else, and that's configuration, not self-hosting (secure access): - Require sign-in. Under the portal's site settings, Redirect anonymous users to sign-in page removes the anonymous default the portal section described. - Sign in with Entra ID, and nothing else. Add Entra ID as the identity provider, then delete the Username and password provider, which "prevents adding users to use username and password authentication" (basic authentication). Entra ID sign-in needs an app registration of its own, which is why the sample keeps the default provider. - Hide what's left with groups. Products visible only to an internal group, and sections of pages restricted to the same group. What configuration can't do on v2 is take the portal off the internet. Even with VNet integration on Standard v2, "the API Management gateway, management plane, and developer portal remain publicly accessible from the internet". Sign-in is the boundary. When self-hosting pays off The portal's source is open, at Azure/api-management-developer-portal, and you can build and host it yourself (self-host). The self-hosted portal is a static site, typically in a storage account's static website, that still uses the managed portal's backend at https://<service name>.developer.azure-api.net for its data; "the content of all portals is stored in the same API Management service". Learn describes it as "full flexibility and extensibility": your own widgets, your own hosting and CDN, your own release cadence, the portal inside a site you already run. It pays off when the limits above are the reason the portal doesn't do its job: you need a custom widget, or the portal has to live inside an existing site. One caveat for v2: the self-host page applies to Basic v2 and Standard v2, and v2 rules out custom widgets in the managed portal, but no Learn page says that custom widgets in a self-hosted portal are supported against a v2 instance. Check that before you plan around it. What you pay: - An Entra app registration. The prerequisites list "permissions to create a Microsoft Entra app registration", next to a storage account, Node.js and the Azure CLI. This is why the article doesn't show a self-hosted portal: the tenant the sample is built in doesn't allow users to create app registrations. - The visibility controls. "Visibility and access controls are supported only in the managed developer portal. They aren't supported in the self-hosted portal." The partners group still decides who can subscribe, but no longer what a page shows. - Upgrades. The managed portal "receives and applies software updates automatically". A self-hosted one is on the release tag you built from (not master, which is daily development) until you rebuild. - Upkeep around it. A SAS token in the configuration that expires, notification email templates that still link to the managed portal's URL until you change them, and support that's limited: "Azure Support's assistance is limited only to the basic setup." If the limit you hit is pages, Standard v2's 50 or a few merged pages is a cheaper fix than owning a portal build. Self-host when you need a capability the managed portal can't have; more headroom alone doesn't justify the build. Conclusion The order API now has a second version and a portal where new partners can find it. Partners who already call /sales/orders haven't noticed anything, because the live API joined its version set as the Original; v2 and its page envelope sit next to it at /sales/v2/orders. New developers find both in the portal, take a free key without asking anyone, and wait for a person only when they want partner limits. The decisions that are hard to undo were made once: path as the scheme, global scope for cors, groups for visibility. The ones that will change often (descriptions, limits, which developer is in partners) are a Bicep change or a click. Those clicks are the problem Part 5 picks up. The products and version set are in Bicep, but group membership, every subscription approved by hand and any policy someone fixed in the Azure portal's editor exist only in the instance. The APIOps extractor pulls an instance's configuration out into files, and the publisher pushes those files into the next environment, so a test instance and production stop drifting apart one portal edit at a time. Is your live API in a version set as the Original, or does it already carry a v1 identifier?
Azure HPC入門：Azure CycleCloud + Slurm + Azure NetApp Filesで作るセキュアなHPCクラスタ環境 !Azure CycleCloud と Slurm、Azure NetApp Files を組み合わせた閉域 HPC/GPU クラスタの全体構成図 Azure CycleCloud 8 と Slurm、Azure NetApp Files（ANF）を組み合わせ、バッチ処理・対話型 JupyterLab・複数ユーザー QoS・ストレージ容量クォータ・オートスケール/常時起動の切り替えをプライベート VNet 内で統合制御する共用クラスタ構成。 なぜこの検証を実施したのか 大学の研究室や企業の R&D 部門で HPC / GPU クラスタをクラウド上に整備する際、「単発の VM を立てて SSH する」だけでは運用が回りません。複数の研究者や学生が同じ環境を共用しようとすると、次のような 7 つの運用上の疑問が挙がります。 1. バッチ処理（Slurm 等）でジョブ投入・キューイング・順次実行ができるか（Q1） 2. JupyterLab 等の対話型（インタラクティブ）操作を同じ Slurm 管理下で両立できるか（Q2） 3. 複数の教員・学生で同一クラスタとホーム領域を安全に共用できるか（Q3） 4. 特定のユーザーが計算資源を長時間独占しないよう、利用時間の上限やジョブ優先順位（QoS）を制御できるか（Q4） 5. 共有ホームディレクトリの容量を特定のユーザーが使い切らないよう、利用者単位でストレージ容量を制限できるか（Q5） 6. ジョブ実行時だけ計算ノードを自動起動・停止（オートスケール）してコンピュート課金を抑えつつ、必要な期間は常時起動（占有）に切り替えられるか（Q6） 7. 将来 H100 や A100 など別世代の GPU VM へ置き換える際、どのように設定変更・パーティション分割を行うか（Q7） これら 7 つの問いに対して、Azure CycleCloud 8 + Slurm + Azure NetApp Files（ANF） の組み合わせでどこまで標準機能として答えられるかを実機構成で検証しました。 > 重要: 本記事の情報は 2026年10月時点の Microsoft Learn 公開情報および Azure CycleCloud 8（Slurm 26.05.4 / cyclecloud-slurm テンプレート）に基づきます。今回の実機検証（TC-01〜TC-07）ではサブスクリプションの GPU クォータ制約を回避してオートスケール・QoS・JupyterLab・ANF クォータの一連の制御を実証するため、計算ノード（hpc パーティション）に Standard_D4s_v6 を割り当てて実測し、GPU VM SKU（Standard_NC24ads_A100_v4 等）の割り当ては CycleCloud GUI の設定画面（TC-08）で確認しています。 --- TL;DR（先に結論） - バッチと対話型（JupyterLab）の両立: Slurm の sbatch でバッチジョブを順次実行できるだけでなく、srun --pty bash で確保した計算ノード上で JupyterLab を起動することで、対話型セッションも同じ Slurm のキュー・時間制限管理下に統合できる。 - 0 台からの自動起動・自動停止と常時起動の切り替え: 通常時は計算ノードを 0 台（計算ノードの VM コンピュート課金停止） にしておき、ジョブ投入をトリガーに自動起動、アイドル時間経過後に自動削除できる。一方で、年間予約や集中利用期間などノードを落としたくない場面では azslurm keep_alive コマンド 1 つで 特定ノードを常時起動（自動停止除外） に切り替えられる。 - 利用時間と優先度は Slurm、容量制限は ANF で役割分担: ジョブの最大実行時間（MaxWall）や優先度（Priority）は Slurm アカウンティング（sacctmgr の QoS）で制御し、/shared/home/$USER のディスク容量上限は Azure NetApp Files の個人ユーザークォータ で制御するのがシンプルで堅牢な設計になる。 - GPU SKU の置き換え: 計算ノードの VM サイズ（SKU）は CycleCloud のパーティション設定で管理されており、OS イメージに NVIDIA GPU ドライバー同梱の AlmaLinux 8.10 HPC（almalinux:almalinux-hpc:8_10-hpc-gen2:latest） 等を採用することで、クラスタ設定の VM サイズ変更またはパーティション追加によって GPU SKU を切り替えられる（ただし GPU 世代を変更する際は CUDA バージョンやコンテナイメージの互換性確認を推奨）。 --- 1. 検証の背景・要件マッピング 共用クラスタに求められる 7 つの要件と、本検証における Azure / Slurm 側の実現方式・検証区分を以下の表に整理します。 | # | 共用環境の要件 | 実現コンポーネント | 制御・設定のポイント | 対応する検証項目・確認方法 | | --- | --- | --- | --- | --- | | Q1 | バッチ処理（ジョブ投入・キューイング・順次実行） | Slurm (sbatch / squeue) | sbatch で計算リソースを要求し、空き枠に応じて順次スケジューリング（GPU 利用時は --gres=gpu:N を指定） | 4-1 (TC-01: 実機検証) | | Q2 | JupyterLab 等の対話型（インタラクティブ）操作 | Slurm (srun) + Windows VDI | srun --pty bash で計算ノードを確保して JupyterLab を起動、Bastion 経由の VDI ブラウザから接続 | 4-4 (TC-04: 実機検証) | | Q3 | 複数の教員・学生・メンバーでのクラスタ共用 | Slurm + ANF (/shared/home) | 複数 OS ユーザー（全ノード共通 UID）でホーム領域を NFS 共有し、ジョブをユーザー別に識別 | 4-5 (TC-05: 実機検証) | | Q4 | 利用時間の上限・ジョブ優先順位の制御 | Slurm Accounting (sacctmgr) | normal（学生想定：上限 1 時間）と high（優先研究想定：上限 24 時間・高優先度）の 2 段階 QoS | 4-6 (TC-06: 実機検証) | | Q5 | 利用者ごとのストレージ容量制限 | Azure NetApp Files (User Quota) | ANF ボリュームの個人ユーザークォータ（UID 単位）で容量上限を設定し quota -s で監視 | 4-7 (TC-07: 実機検証) | | Q6 | 従量課金の自動起動/停止と、占有（常時起動）の併用 | CycleCloud Autoscale (azslurm) | ジョブ連動の 0 台↔N 台オートスケールと、azslurm keep_alive による常時起動ノードの固定 | 4-2, 4-3 (TC-02, TC-03: 実機検証) | | Q7 | H100 / A100 等、別 GPU SKU への置き換え・混在 | CycleCloud クラスタテンプレート | Node Array の VM サイズ変更、または hpc / gpu パーティションの分割 | 4-8 (TC-08: GUI 設定画面確認) | > 出典: Azure CycleCloud の概要 / Azure CycleCloud での Slurm の使用 --- 2. アーキテクチャと設計のポイント 2-1. 全体アーキテクチャ図 本検証で構築する環境の全体アーキテクチャを下図に示します。すべてのクラスタ VM からパブリック IP を排除し、Azure Bastion を唯一の入り口とする閉域ネットワーク構成としています。 !Azure CycleCloud + Slurm + Azure NetApp Files による閉域 GPU HPC クラスタ構成図 構成図内の主要コンポーネント、起動形態（常時起動かオートスケールか）、および役割は以下のとおりです。 | レイヤー / 境界 | コンポーネント | 構成・サイズ例 | 起動形態 | 役割と設計上の意図 | | --- | --- | --- | --- | --- | | アクセス経路 (AzureBastionSubnet) | Azure Bastion (bastion-hpc01) | Standard SKU | 常時稼働（PaaS） | クラスタ VM にパブリック IP を持たせず、ポータルや CLI 経由で安全に RDP / SSH 接続を提供する踏み台サービス。 | | クラスタサブネット (subnet-cluster01 10.0.1.0/24) | Azure NAT Gateway (natgw-hpc01) | Standard SKU | 常時稼働（PaaS） | 2026年3月末以降のプライベートサブネット既定化（defaultOutboundAccess: false）に対応し、閉域 VM から Azure Resource Manager（management.azure.com）やパッケージリポジトリへの送信通信を提供する。 | | クラスタサブネット (subnet-cluster01 10.0.1.0/24) | Windows VDI Node (vdi-win01) | Standard_D4s_v6 (Windows Server) | 利用時のみ起動（手動停止可） | 閉域 VNet 内の CycleCloud Web UI（HTTPS）および計算ノード上の JupyterLab（HTTP 8888）へブラウザアクセスするための作業用端末。 | | クラスタサブネット (subnet-cluster01 10.0.1.0/24) | CycleCloud Server (cc-server01) | Standard_D4s_v6 (AlmaLinux / RHEL) | クラスタ稼働中は常時起動（1台） | クラスタのテンプレート管理・VM プロビジョニング・オートスケールを司るオーケストレーター（インフラ管理サーバー）。Slurm のジョブスケジューラそのものではなく、裏方で Azure VM/VMSS を作成・削除する。 | | クラスタサブネット (subnet-cluster01 10.0.1.0/24) | Slurm Head Node (scheduler) | Standard_D4s_v6 (AlmaLinux 8.10 HPC) | クラスタ稼働中は常時起動（1台） | Slurm のスケジューラーノード兼ログインノード。slurmctld（ジョブスケジューラ）および slurmdbd（アカウンティング）を実行し、学生・研究者が SSH ログインして sbatch / srun を投入する窓口となる。ジョブが入ると azslurm 経由で CycleCloud Server の API を呼び出して計算ノードの起動を要求する。 | | クラスタサブネット (subnet-cluster01 10.0.1.0/24) | Compute Nodes (hpc / gpu / htc) | GPU / 汎用 VM SKU（初期 0 台、ジョブ連動） | オートスケール（0台↔N台）<br>※keep_alive で常時起動化も可 | AlmaLinux 8.10 HPC イメージで起動する計算ノード群。ジョブがない時は 0 台（VM コンピュート課金ゼロ） で待機し、ジョブ投入時に自動起動、アイドル時に自動削除される。 | | ANF 委任サブネット (subnet-anf01 10.0.253.0/28) | Azure NetApp Files (anf-stg / vol01) | Standard サービスレベル / 4 TiB プール | 常時稼働（PaaS） | /shared/home/$USER 等の共有 NFS ボリュームを提供し、UID 単位のユーザークォータで容量上限を強制する。 | | 認証・保管 (Identity & Storage) | User Assigned Managed Identity (mngdid-rg-hpc01) | RG Contributor + Storage Blob Data Contributor + Sub Reader | — | CycleCloud サーバーおよびクラスタに割り当て、シークレット管理なしに VM 作成・Locker 読み書き・SKU カタログ取得を行う。 | | 認証・保管 (Identity & Storage) | CycleCloud Locker (stgacctcc01) + Private Endpoint | StorageV2 + privatelink.blob.core.windows.net | 常時稼働（PaaS） | CycleCloud が cluster-init スクリプトや Slurm パッケージを配置する Blob ストレージ。Private Endpoint 経由で閉域アクセスする。 | > 補足（CycleCloud Server と Slurm Head Node の違い・常時起動の範囲）: > - CycleCloud Server はスケジューラーノードではありません。CycleCloud Server（cc-server01）は Azure リソースを管理するオーケストレーターであり、Slurm のスケジューラー（兼ユーザーのログインノード）は CycleCloud が起動する Slurm Head Node（scheduler） です。 > - ジョブをいつでも受け付けられるようにするため、クラスタ稼働中は CycleCloud Server と Slurm Head Node (scheduler) の 2 台（安価な汎用 VM）は常時起動させておき、高価な Compute Nodes（GPU 計算ノード）だけを 0 台↔N 台でオートスケールさせるのが基本運用です（長期休暇等でクラスタ自体を使わない期間は、CycleCloud 上でクラスタを Terminate して scheduler も削除し、cc-server01 自体も VM 停止できます）。 2-2. なぜこの構成なのか（3 つの設計判断） 1. すべての計算・管理 VM からパブリック IP を排除（Bastion + Windows VDI + NAT Gateway 方式） 大学・研究機関のセキュリティポリシーでは、計算ノードや管理サーバーへインターネットから直接インバウンド接続させない構成が強く求められます。各 VM にはパブリック IP を付与せず、インバウンドは Azure Bastion 経由のみとし、アウトバウンド通信（ARM API やパッケージ取得）はサブネットに紐付けた Azure NAT Gateway で明示的に制御します。CycleCloud の Web UI や計算ノード上で起動する JupyterLab（ポート 8888）へは、同一 VNet 内に配置した Windows VDI ノードのブラウザからプライベート IP でアクセスすることで、SSH ポートフォワーディングなしに直感的な GUI 操作を実現できます。 2. HPC / GPU ドライバー導入済みの AlmaLinux 8.10 HPC イメージ オートスケールで計算ノードが動的に起動・破棄される環境では、起動のたびにドライバーやライブラリをビルドしているとジョブ開始までの待ち時間が増加します。Azure Marketplace で提供されている almalinux:almalinux-hpc:8_10-hpc-gen2:latest（または microsoft-dsvm:ubuntu-hpc:2404:latest）は、NVIDIA GPU ドライバー・Mellanox OFED（InfiniBand ドライバー）・主要な HPC ライブラリがあらかじめ組み込まれているため、GPU SKU を選択した際もプロビジョニング完了後すぐにジョブを実行できます。 3. 「計算リソース制御（Slurm）」と「ストレージ容量制御（ANF）」の分離 Slurm は CPU / メモリ / GPU（GRES）の割り当て時間や優先度を制御するのには優れていますが、NFS 上のホームディレクトリ容量の制限はファイルシステム側の責務です。Azure NetApp Files（ANF）のユーザークォータ機能を組み合わせることで、Linux 標準の UID に基づいてストレージ側で確実な容量上限をかけられます。 > 出典: Azure HPC/AI VM イメージ / Azure NetApp Files の既定および個別のユーザーとグループのクォータについて 2-3. 利用者（学生・研究者）と管理者のアカウント設計・アクセス導線 「学生が使う場合、どこにアカウントを作り、どのように環境へ入っていくのか（CycleCloud に入るのか？ VDI 経由なのか？）」という運用イメージを整理します。 結論として、学生は「CycleCloud の管理画面」には一切入りません。学生が使うのは 「Slurm ヘッドノード（scheduler）への SSH」 と、JupyterLab を使う場合の 「VDI のブラウザ」 だけです。 | 誰が（ロール） | 何をするか | どこに入るか（接続経路） | どこにアカウントを作るか | | --- | --- | --- | --- | | 管理者<br>（教職員・インフラ担当） | クラスタ起動/停止・GPU SKU 変更・クォータ設定 | Bastion → VDI（RDP）→ ブラウザで CycleCloud 管理画面（cc-server01） | CycleCloud 管理画面 ＋ Azure RBAC | | 学生・研究者<br>（バッチジョブ実行） | sbatch でジョブ投入・結果確認 | Bastion（または学内 VPN）→ Slurm ヘッドノード（scheduler）へ直接 SSH<br>※VDI や CycleCloud は使わない | Linux OS ユーザー（共通 UID）＋ Slurm DB（sacctmgr） | | 学生・研究者<br>（JupyterLab 対話利用） | ブラウザで Notebook を対話実行 | Bastion → VDI（RDP） → scheduler へ SSH して srun で Jupyter 起動 → VDI 内ブラウザで接続 | Linux OS ユーザー ＋ Slurm DB ＋ VDI ログイン用アカウント | > ポイント（Linux アカウントの一元管理）: > オートスケールで動的に起動する計算ノードでも同じ UID / ホーム（/shared/home/$USER）でジョブが動くよう、実運用では CycleCloud 8 の組み込みユーザー管理や Active Directory / Microsoft Entra Domain Services / LDAP 連携で全ノード共通の UID を一元管理します。 学生が Bastion 経由で入る場合、Azure の RBAC 付与は必要か？ Azure Bastion を入り口にする場合、接続方式によって学生への Azure RBAC 付与が「必要なケース」と「不要にできるケース」に分かれます。 | 接続方式 | 学生への Azure RBAC | 必要な権限・運用の特徴 | | --- | --- | --- | | A. Azure Portal / Azure CLI から Bastion 接続<br>（標準の Bastion SSH / RDP / az network bastion tunnel） | 必要 | 学生の Entra ID に対し、①対象 VM（scheduler / vdi-win01）、②対象 VM の NIC、③Azure Bastion リソースの 3 点に対する Reader（閲覧者）ロールが必要。 | | B. Bastion の「共有可能リンク（Shareable Link）」を利用<br>（Bastion Standard SKU 以上の機能） | 不要 | 管理者が発行した専用 URL を学生に配布する方式。学生は Azure Portal 権限（RBAC）や Entra ID なしで、ブラウザから URL を開いて Linux の SSH 秘密鍵（または RDP 資格情報）だけでログイン可能。 | | C. 学内 VPN / ExpressRoute 経由で直接接続<br>（大学キャンパス網と Azure VNet を接続する場合） | 不要 | Bastion を介さず手元 PC からプライベート IP（10.0.1.7）へ直接 SSH できるため、Azure RBAC は一切不要（Linux アカウントと SSH 鍵のみで完結）。 | > 出典: Azure Bastion FAQ - 仮想マシンにアクセスするために必要なロールはありますか? / Bastion の共有可能リンクを作成する --- 3. 環境構築手順（ハンズオン準拠） ここからは、実際に Azure CLI で基盤リソースを展開し、CycleCloud 上で Slurm クラスタを構築する手順を示します。 > 注意: CycleCloud クラスタを展開する際、VNet・ANF・CycleCloud サーバー・計算ノードは同一リージョンに配置します（本検証では変数 $LOCATION="southeastasia" で統一しています）。 3-1. 事前準備（RG・マネージド ID・VNet・NAT Gateway・閉域 Storage・Bastion） まず、リソースグループ、CycleCloud サーバーに付与するユーザー割り当てマネージド ID、仮想ネットワーク（3 つのサブネット＋NAT Gateway）、および Private Endpoint 付きの CycleCloud Locker 用ストレージアカウントを作成します。 > 重要（2026年3月31日以降の仕様変更）: Azure では 2026年3月31日以降の API バージョンで新規作成されるサブネットが既定でプライベート サブネット（defaultOutboundAccess: false）になりました。パブリック IP を持たない cc-server01 や scheduler から Azure Resource Manager（management.azure.com）や GitHub へアクセスするため、subnet-cluster01 には Azure NAT Gateway を必ず関連付けます。 <details><summary>01-infra-prereq.sh: RG・マネージドID・VNet・NAT Gateway・閉域Storage・Bastionの作成（クリックで展開）</summary> # 01-infra-prereq.sh # リージョンとリソース名の定義（全リソースを同一リージョンに統一する） RG="rg-hpc01-sea" LOCATION="southeastasia" VNET="vnet-hpc01" MNGD_ID="mngdid-rg-hpc01" STG_ACCT="stgacctcc01$(openssl rand -hex 2)" # 1. リソースグループの作成 az group create -n "$RG" -l "$LOCATION" # 2. CycleCloud 用ユーザー割り当てマネージド ID の作成と RBAC ロール付与 az identity create -n "$MNGD_ID" -g "$RG" -l "$LOCATION" PRINCIPAL_ID=$(az identity show -n "$MNGD_ID" -g "$RG" --query principalId -o tsv) RG_ID=$(az group show -n "$RG" --query id -o tsv) SUB_ID=$(az account show --query id -o tsv) az role assignment create --assignee-object-id "$PRINCIPAL_ID" --assignee-principal-type ServicePrincipal --role "Contributor" --scope "$RG_ID" az role assignment create --assignee-object-id "$PRINCIPAL_ID" --assignee-principal-type ServicePrincipal --role "Storage Blob Data Contributor" --scope "$RG_ID" az role assignment create --assignee-object-id "$PRINCIPAL_ID" --assignee-principal-type ServicePrincipal --role "Reader" --scope "/subscriptions/$SUB_ID" # 3. 仮想ネットワークと 3 つのサブネット（cluster + NAT Gateway / anf委任 / Bastion）の作成 az network nsg create -g "$RG" -n "nsg-cluster01" -l "$LOCATION" az network public-ip create -g "$RG" -n "pip-natgw-hpc01" -l "$LOCATION" --sku Standard az network nat gateway create -g "$RG" -n "natgw-hpc01" -l "$LOCATION" --public-ip-addresses "pip-natgw-hpc01" --idle-timeout 10 az network vnet create -g "$RG" -n "$VNET" -l "$LOCATION" \ --address-prefixes 10.0.0.0/16 \ --subnet-name "subnet-cluster01" --subnet-prefixes 10.0.1.0/24 \ --network-security-group "nsg-cluster01" --nat-gateway "natgw-hpc01" # ANF 用サブネット（Microsoft.NetApp/volumes へ委任） az network vnet subnet create -g "$RG" --vnet-name "$VNET" -n "subnet-anf01" \ --address-prefixes 10.0.253.0/28 --delegations "Microsoft.NetApp/volumes" # Azure Bastion 用サブネット az network vnet subnet create -g "$RG" --vnet-name "$VNET" -n "AzureBastionSubnet" \ --address-prefixes 10.0.254.0/26 # 4. CycleCloud Locker 用ストレージアカウントと Private Endpoint の作成 az storage account create -n "$STG_ACCT" -g "$RG" -l "$LOCATION" --sku Standard_LRS --kind StorageV2 --allow-blob-public-access false STG_ID=$(az storage account show -n "$STG_ACCT" -g "$RG" --query id -o tsv) az network private-dns zone create -g "$RG" -n "privatelink.blob.core.windows.net" az network private-dns link vnet create -g "$RG" -z "privatelink.blob.core.windows.net" -n "link-$VNET" -v "$VNET" -e false az network private-endpoint create -g "$RG" -n "pe-$STG_ACCT-blob" -l "$LOCATION" \ --vnet-name "$VNET" --subnet "subnet-cluster01" \ --private-connection-resource-id "$STG_ID" --group-id "blob" --connection-name "conn-blob" az network private-endpoint dns-zone-group create -g "$RG" --endpoint-name "pe-$STG_ACCT-blob" \ -n "default" --private-dns-zone "privatelink.blob.core.windows.net" --zone-name "blob" # 5. Azure Bastion の展開 az network public-ip create -g "$RG" -n "pip-bastion-hpc01" -l "$LOCATION" --sku Standard az network bastion create -g "$RG" -n "bastion-hpc01" -l "$LOCATION" \ --public-ip-address "pip-bastion-hpc01" --vnet-name "$VNET" --sku Standard --enable-tunneling true </details> 作成後の検証コマンド: az network private-endpoint show -g "$RG" -n "pe-$STG_ACCT-blob" --query "provisioningState" -o tsv # Succeeded が返ることを確認 3-2. Azure NetApp Files（ANF）の展開と共有ボリューム作成 続いて、/shared/home/$USER としてマウントする Azure NetApp Files のアカウント、容量プール（4 TiB / Standard）、および NFSv3 ボリューム（vol01）を作成します。 <details><summary>02-create-anf.sh: ANFアカウント・容量プール・NFSボリュームの作成（クリックで展開）</summary> # 02-create-anf.sh ANF_ACCT="anf-stg" ANF_POOL="anf-stg-pool01" ANF_VOL="vol01" # 1. NetApp アカウントと容量プール（4 TiB, Standard）の作成 az netappfiles account create -g "$RG" -n "$ANF_ACCT" -l "$LOCATION" az netappfiles pool create -g "$RG" --account-name "$ANF_ACCT" -n "$ANF_POOL" -l "$LOCATION" \ --size 4 --service-level Standard # 2. NFSv3 ボリューム（100 GiB、subnet-anf01 に配置）の作成 az netappfiles volume create -g "$RG" --account-name "$ANF_ACCT" --pool-name "$ANF_POOL" -n "$ANF_VOL" -l "$LOCATION" \ --service-level Standard --usage-threshold 100 --file-path "vol01" \ --vnet "$VNET" --subnet "subnet-anf01" --protocol-types NFSv3 \ --rule-index 1 --allowed-clients "10.0.0.0/16" --unix-read-write true --has-root-access true </details> 作成後、Slurm テンプレートに設定する ANF のマウント先 IP アドレスを確認します。 az netappfiles volume show -g "$RG" --account-name "$ANF_ACCT" --pool-name "$ANF_POOL" -n "$ANF_VOL" --query "mountTargets[0].ipAddress" -o tsv # 例: 10.0.253.4 が返ることを確認 3-3. CycleCloud サーバーと Windows VDI ノードの展開 CycleCloud サーバー VM（パブリック IP なし・ユーザー割り当てマネージド ID アタッチ）と、ブラウザ操作用の Windows VDI ノードを subnet-cluster01 に作成します。 <details><summary>03-create-cyclecloud-and-vdi.sh: CycleCloudサーバーとWindows VDIの展開（クリックで展開）</summary> # 03-create-cyclecloud-and-vdi.sh ID_RES_ID=$(az identity show -n "$MNGD_ID" -g "$RG" --query id -o tsv) # Marketplace の CycleCloud イメージ規約に同意 az vm image terms accept --urn "azurecyclecloud:azure-cyclecloud:cyclecloud8-gen2:latest" # 1. CycleCloud サーバー VM の作成（パブリック IP なし、マネージド ID 付与） az vm create -g "$RG" -n "cc-server01" -l "$LOCATION" \ --image "azurecyclecloud:azure-cyclecloud:cyclecloud8-gen2:latest" \ --size "Standard_D4s_v6" \ --vnet-name "$VNET" --subnet "subnet-cluster01" --public-ip-address "" \ --private-ip-address "10.0.1.6" --nsg "" \ --assign-identity "$ID_RES_ID" \ --admin-username "azureuser" --ssh-key-values "@ssh_keys/id_rsa.pub" # 2. Windows VDI ノードの作成（CycleCloud Web UI / JupyterLab アクセス用） az vm create -g "$RG" -n "vdi-win01" -l "$LOCATION" \ --image "MicrosoftWindowsServer:WindowsServer:2022-datacenter-g2:latest" \ --size "Standard_D4s_v6" \ --vnet-name "$VNET" --subnet "subnet-cluster01" --public-ip-address "" --nsg "" \ --admin-username "azureuser" --admin-password "<StrongPassword123!>" </details> 3-4. Slurm クラスタテンプレート（cyclecloud-slurm）の導入と GPU 設定 Azure Bastion 経由で vdi-win01 に RDP 接続し、ブラウザから CycleCloud Web UI へアクセスして初期セットアップ（Site Name、Managed Identity 認証、Storage Locker stgacctcc01 の選択、管理者 SSH 公開鍵の登録）を完了します。 !CycleCloud の Subscriptions 設定画面における Managed Identity 認証と Storage Locker の登録 CycleCloud の Subscriptions 設定画面。Orchestrator Configuration と Locker Configuration の双方でユーザー割り当てマネージド ID（mngdid-rg-hpc01）を指定し、閉域 Storage Account（stgacctcc011402）を Locker として紐付けます。 その後、CycleCloud サーバーへ SSH ログインし、GitHub から cyclecloud-slurm テンプレートを取得してインポートします（CycleCloud 8 標準搭載の Slurm テンプレートをそのまま利用することも可能です）。 <details><summary>04-import-slurm-template.sh: cyclecloud-slurm 3.0.10 の取得とインポート（クリックで展開）</summary> # 04-import-slurm-template.sh（cc-server01 上で実行） mkdir -p ~/cyclecloud_projects && cd ~/cyclecloud_projects wget https://github.com/Azure/cyclecloud-slurm/archive/refs/tags/3.0.10.zip unzip 3.0.10.zip ln -s cyclecloud-slurm-3.0.10 cyclecloud-slurm cd cyclecloud-slurm/templates # テンプレート名を Slurm-poc01 としてインポート cp -p slurm.txt slurm.txt.org sed -i 's/\[cluster Slurm\]/[cluster Slurm-poc01]/' slurm.txt cyclecloud import_template -f slurm.txt </details> CycleCloud GUI で新規 Slurm クラスタ（Slurm-poc01）を作成する際、以下の 4 点を設定します。 | 設定タブ | パラメータ | 設定値 | 理由 | | --- | --- | --- | --- | | Required Settings | Scheduler VM Type / HPC VM Type / GPU VM Type | Scheduler: Standard_D4s_v6 / HPC: Standard_D4s_v6 / GPU: Standard_NC24ads_A100_v4 等 | 計算ノード（hpc / gpu パーティション）に用途別の VM サイズを割り当てるため | | Required Settings | Autoscale / Max Nodes | Start and stop execute instances automatically を ON（初期起動台数は 0） | ジョブがない時は 0 台に保ち、投入時のみスケールアウトさせるため | | Network Attached Storage | File-system Mount for /shared | Use Builtin NFS を OFF → External NFS / IP 10.0.253.4 / Export Path /vol01 | ANF ボリュームを全ノード共通の /shared（/shared/home/$USER）としてマウントするため | | Advanced Settings | Managed Id / OS (Alma Linux 8) | Managed Id: mngdid-rg-hpc01 / OS: Alma Linux 8 (almalinux:almalinux-hpc:8_10-hpc-gen2:latest) | Locker アクセス用マネージド ID と NVIDIA GPU ドライバー同梱の HPC イメージを指定し、完全閉域（Return Proxy・Public Head Node オフ）で起動するため | !CycleCloud の Required Settings タブにおける Scheduler / HPC / GPU VM Type と Autoscale の設定 Required Settings タブの設定画面。Scheduler VM Type・HPC VM Type・GPU VM Type をそれぞれ指定し、Autoscale を有効化します。 !CycleCloud の Network Attached Storage タブにおける External NFS（Azure NetApp Files）の設定 Network Attached Storage タブの設定画面。/shared の Use Builtin NFS のチェックを外して External NFS を選択し、ANF ボリュームのマウント IP（10.0.253.4）と Export Path（/vol01）を指定します（下部の Additional Filesystem Mount で二重に /shared を指定しないよう注意します）。 !CycleCloud の Advanced Settings タブにおける Managed Identity と AlmaLinux 8 HPC イメージの設定 Advanced Settings タブの設定画面。Managed Id に rg-hpc01-sea/mngdid-rg-hpc01 を選択し、OS に Alma Linux 8（almalinux:almalinux-hpc:8_10-hpc-gen2:latest）を指定します。 > 出典: GitHub - Azure/cyclecloud-slurm --- 4. 7つの運用シナリオ実機検証（TC-01〜TC-08） クラスタを開始（Start）すると、約 6 分で Slurm ヘッドノード（scheduler）が Ready（緑色）となり、計算ノードは 0 台 の状態で待機します。ここから冒頭の 7 つの質問（Q1〜Q7）に対応する 8 つのテストケースを順に検証します。 !Slurm クラスタ起動直後の CycleCloud 画面（scheduler のみ稼働、計算ノードは 0 台） クラスタ起動完了直後の状態。scheduler ノード（1 台・4 コア）のみが稼働（緑色）しており、計算ノードは 0 台で課金が発生していないことが確認できます。 4-1. バッチ処理とジョブの順次実行（TC-01 / Q1） Slurm ヘッドノード（10.0.1.7）へ SSH ログインし、バッチジョブスクリプトを作成して sbatch で 3 本連続投入します。 <details><summary>gpu-job01.sh: Slurmバッチジョブスクリプト（クリックで展開）</summary> #!/bin/bash #SBATCH --partition=hpc #SBATCH --job-name=gpu-job01 #SBATCH --nodes=1 #SBATCH --ntasks=1 #SBATCH --cpus-per-task=4 #SBATCH --time=00:10:00 #SBATCH --output=slurm-%x-%j.out # 1. ジョブ開始時刻と、割り当てられた計算ノードのホスト名（例: slurm-poc01-hpc-1）を出力 echo "=== Job Start: $(date) on $(hostname) ===" # 2. GPU 搭載ノードなら nvidia-smi を出力し、GPU 非搭載の汎用 VM なら代わりに lscpu で CPU 情報を出力 nvidia-smi 2>/dev/null || lscpu | grep -E "Model name|Socket|Thread|NUMA|CPU\(s\):" # 3. squeue で実行中（R）状態や複数ジョブの挙動を観察できるよう、模擬処理として 120 秒間維持 sleep 120 # 4. ジョブ終了時刻を出力 echo "=== Job End: $(date) ===" </details> スクリプト後半の 4 行は以下の意味を持っています。 - echo "=== Job Start: $(date) on $(hostname) ===": ジョブの開始日時と、実際に処理が走っているホスト名（ヘッドノードではなく計算ノード slurm-poc01-hpc-1 等であることの証跡）を出力します。 - nvidia-smi 2>/dev/null || lscpu | grep -E ...: まず nvidia-smi で GPU の認識状態を表示し、もし GPU 非搭載の汎用 VM（今回の Standard_D4s_v6 代替検証環境など）で nvidia-smi が失敗した場合（||）は、代わりに lscpu で CPU のモデル名・コア数・NUMA 構成を抜粋して出力します。GPU ノード・CPU ノードのどちらでも同じスクリプトのまま動作確認できる書き方です。 - sleep 120: 本来の機械学習や数値計算の代わりに 120 秒間待機します。処理が一瞬で終わると squeue 上の R（Running）状態や複数ジョブのキューイングを確認できないために入れています。 - echo "=== Job End: $(date) ===": 120 秒経過後のジョブ終了日時を記録します。 <details><summary>ジョブの3本連続投入とキュー確認コマンド（クリックで展開）</summary> sbatch ~/gpu-job01.sh sbatch ~/gpu-job01.sh sbatch ~/gpu-job01.sh squeue </details> これにより、計算資源の空き枠に応じてジョブが割り当てられ、完了後に結果ログを取得できます。空きノードが複数あれば 3 本は並行実行されます。順次実行を確認する場合は、利用可能な計算資源を制限した条件で、先行ジョブの終了後に後続ジョブが開始することを確認します。 実行結果はどこで確認するか 結果を確認するのは CycleCloud 管理画面ではなく、ジョブを投入したユーザーで SSH 接続した scheduler 上です。 上のスクリプトでは #SBATCH --output=slurm-%x-%j.out を指定しているため、%x がジョブ名、%j がジョブ ID に置き換わり、例えばジョブ ID 12 の出力ファイルは slurm-gpu-job01-12.out になります。標準エラーも、別途 --error を指定しなければ同じファイルへ出力されます。 相対パスの出力先は、通常 sbatch を実行したときの作業ディレクトリ です。sbatch ~/gpu-job01.sh と指定しても、スクリプトの保存場所が出力先になるわけではありません。/shared/home/$USER に移動して投入した場合に、その共有ホームへ保存されます。--chdir を指定した場合は、そのディレクトリが基準です。 <details><summary>ログの場所・実行中ジョブ・終了状態を確認するコマンド（クリックで展開）</summary> # ジョブを投入したディレクトリで実行。12 は実際のジョブ ID に置換 pwd ls -lt slurm-gpu-job01-*.out cat slurm-gpu-job01-12.out # 自分の待機中・実行中ジョブ（終了したジョブは通常表示されない） squeue -u "$USER" # 今日のジョブ履歴と、特定ジョブの終了状態・時刻・実行ノード sacct -S today -u "$USER" -X --format=JobID,JobName,State,ExitCode,Start,End,NodeList sacct -j 12 --format=JobID,State,ExitCode,NodeList,Elapsed,ReqTRES,AllocTRES # コントローラーに記録が残っていれば WorkDir・StdOut・StdErr を確認できる scontrol show job 12 </details> scontrol show job は古い終了ジョブでは記録が消えていることがあります。その場合は sacct で履歴を確認し、投入時の作業ディレクトリからログを探します。共有領域に保存されたログは、計算ノードが自動削除された後も scheduler から確認できます。 | 確認対象 | 見る場所・項目 | 判定のポイント | | --- | --- | --- | | 待機・実行状況 | squeue の ST と NODELIST(REASON) | PD は待機、CF は起動・構成中、R は実行中 | | 終了状態 | sacct の State / ExitCode | COMPLETED / 0:0 はスクリプトの正常終了。ただし内部の全コマンド成功を保証しない | | アプリケーションの出力 | slurm-gpu-job01-<JOBID>.out | 開始・終了時刻、実行ホスト名、計算結果、エラーを読む | | 順次実行 | 3 本それぞれの Start / End | 同じ制約資源を使う先行ジョブの終了後に、後続ジョブが開始しているか確認 | D4s_v6 で NVIDIA-SMI のエラーが出る場合 2026年11月4日の再実行では、Standard_D4s_v6 の計算ノード上で次の出力を確認しました。 === Job Start: Wed Nov 4 07:30:05 UTC 2026 on slurm-poc01-hpc-1 === NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running. Standard_D4s_v6 は GPU 非搭載の VM です。 この構成では GPU 動作確認はできませんが、CPU 情報の出力と 120 秒後の終了により、計算ノード上でバッチが動いたことは確認できます。上のスクリプトは nvidia-smi 失敗時に CPU 情報へ切り替えるため、Slurm 上で COMPLETED になっていても、GPU 利用に成功したことにはなりません。GPU 搭載 VM でもドライバー障害時には同様の切り替えが起きるため、このスクリプトはバッチの動作確認用であり、GPU の合否判定用ではありません。 GPU の検証では、GPU 搭載 VM と適切な NVIDIA ドライバーを用意し、#SBATCH --gres=gpu:1 を追加して GPU を要求したうえで、nvidia-smi の成功を別途確認してください。 > 出典: Slurm sbatch（出力先・ファイル名の指定） / Slurm sacct（実行履歴） / Azure Dsv6 シリーズの仕様 4-2. 計算ノードの自動起動・自動停止（TC-02 / Q6） CycleCloud + Slurm の大きなメリットが、「ジョブがない時は計算ノード 0 台、ジョブが入ると自動起動、終われば自動削除」 というオートスケール動作です。 1. ジョブ投入前: CycleCloud GUI 上で計算ノードが 0 台（sinfo の STATE が idle~：未起動待機）であることを確認します。 2. ジョブ投入時: sbatch ~/gpu-job01.sh を 3 本実行すると、sinfo の STATE が alloc#（プロビジョニング中）、squeue の STATE が CF（Configuring）になり、CycleCloud 側で計算ノード VMSS（Slurm-poc01-hpc-[1-3]）の作成が自動的に開始されます。 !sbatch 投入により CycleCloud 上で計算ノード 3 台が自動起動し squeue が CF 状態になっている画面 ジョブ投入直後の画面（左：CycleCloud Web UI、右：ヘッドノード SSH ターミナル）。sbatch をトリガーに hpc ノード 3 台のプロビジョニングが自動開始され、squeue 上で CF（Configuring）として表示されています。 3. ノード起動完了とジョブ実行: 約 7〜8 分で VM が起動して Slurm クラスタに参加すると、STATE が R（Running）へ遷移してジョブが実行され、slurm-gpu-job01-1.out に計算ノード（slurm-poc01-hpc-1）の実行結果が出力されます。 !計算ノードの起動完了後にジョブが実行完了し sacct と結果ログが出力された画面 計算ノード起動完了後の実行結果。sacct でジョブが COMPLETED となり、slurm-gpu-job01-1.out に計算ノード slurm-poc01-hpc-1 上での出力が記録されています。 4. ジョブ終了後の自動停止: ジョブ完了後、Slurm の SuspendTime（既定 300 秒）が経過すると、計算ノード VM は自動的に Deallocate（削除）され、再び 0 台（課金ゼロ） に戻ります。 > ポイント: GPU VM の GPU 枚数が Slurm の GRES に正しく認識されない場合（Requested node configuration is not available となる場合）は、ヘッドノードの /opt/azurehpc/slurm/autoscale.json に slurm_gpus を明示定義して azslurm scale と azslurm buckets を実行することで即時反映できます（詳細は「5. ハマったポイント」参照）。 4-3. azslurm keep_alive による計算ノードの常時起動（TC-03 / Q6） 「通常はオートスケールで節約したいが、特定の研究期間や年間予約枠のノードは毎回数分の起動待ちをせずに常時起動（占有）させておきたい」という要件（Q6）に対しては、クラスタを作り直すことなく azslurm keep_alive コマンドでノード単位に制御できます。 <details><summary>05-keep-alive.sh: 特定ノードを自動停止対象から除外して常時起動化する手順（クリックで展開）</summary> # 1. 対象の計算ノードが起動している状態で、ヘッドノードにて keep_alive を設定 sudo azslurm keep_alive --set --node-list Slurm-poc01-hpc-1 # 2. slurmctld を再起動し、azslurm scale で反映 sudo systemctl restart slurmctld sudo azslurm scale cat /etc/slurm/keep_alive.conf # （常時起動を解除して再びオートスケールに戻す場合） sudo azslurm keep_alive --remove --node-list Slurm-poc01-hpc-1 sudo systemctl restart slurmctld sudo azslurm scale </details> !azslurm keep_alive コマンドにより Slurm-poc01-hpc-1 を自動停止対象から除外（SuspendExcNodes）した画面 azslurm keep_alive --set の実行画面。CycleCloud GUI 側の Status Message が KeepAlive に変わり、/etc/slurm/keep_alive.conf の SuspendExcNodes=Slurm-poc01-hpc-1 によってアイドル時も自動停止されなくなります。 これにより、同じ 1 つの Slurm クラスタの中で「必要なときだけ起動する従量ノード」と「常時起動しておく固定ノード」を柔軟に使い分けられることが確認できます。 > 出典: Azure CycleCloud Slurm - パーティションやノードをオートスケールから除外する (keep_alive) 4-4. Slurm 管理下での対話型利用（JupyterLab）（TC-04 / Q2） 「バッチだけでなく JupyterLab で対話的に試行錯誤したい」という要件（Q2）に対し、計算ノードへ直接勝手に入るのではなく、Slurm のインタラクティブジョブ（srun）として計算リソースと利用時間を確保した上で JupyterLab を起動します。 <details><summary>srunによる対話ジョブ取得とJupyterLab起動コマンド（クリックで展開）</summary> # 1. ヘッドノードから最大 1 時間を指定して計算ノードの対話シェルを取得 srun --partition=hpc --time=01:00:00 --pty bash # 2. 割り当てられた計算ノード（slurm-poc01-hpc-1）上で IP を確認し、JupyterLab を起動 hostname -I ~/.local/bin/jupyter lab --no-browser --ip=0.0.0.0 --port=8888 </details> 起動時に出力される URL（http://10.0.1.8:8888/lab）を、Azure Bastion 経由で接続している Windows VDI ノード（vdi-win01）のブラウザ で開きます。 !srun で確保した計算ノード上で JupyterLab を起動し Windows VDI のブラウザから接続して実行している画面 Windows VDI 内のブラウザから計算ノード（10.0.1.8:8888）の JupyterLab に接続し、Notebook セル上で !hostname（slurm-poc01-hpc-1）と !lscpu を対話実行している様子。右側のターミナルでは srun によってセッションが Slurm ジョブとして管理されています。 この方式のメリットは以下の 2 点です。 - Notebook セルから対話的にコードを実行でき、ファイルは /shared/home/$USER（ANF）に直接保存される。 - 対話型で利用中のノードも squeue 上に 1 ジョブとして可視化され、指定した --time（または QoS の上限時間）が来れば自動的に解放されるため、「誰かが Jupyter を開いたまま放置して計算ノードがずっと空かない」という共用クラスタ特有の課題を防げる。 4-5. 複数ユーザーでのクラスタ共用（TC-05 / Q3） 共用環境（Q3）の検証として、全ノード共通の UID を持つ 2 名のユーザー（user01：学生想定、user02：教員想定）を作成し、ホームディレクトリを ANF 上の /shared/home/$USER に配置します。 # 共通 UID で user01 / user02 を作成し、ホームを /shared/home 配下に配置 sudo useradd -u 20003 -d /shared/home/user01 -m user01 sudo useradd -u 20002 -d /shared/home/user02 -m user02 両ユーザーから同時にジョブを投入すると、squeue 上で USER 列によって誰のジョブかが明確に識別され、計算ノード側でも同じ UID で /shared/home/user01・/shared/home/user02 にアクセスできることを確認できます。 !user01 と user02 の複数ユーザーからジョブを投入し squeue でユーザー別に管理されている画面 user01（JOBID 4）と user02（JOBID 5）の双方からジョブを投入した状態。squeue の USER 列でユーザーごとにジョブが識別され、CycleCloud 側で 2 台の計算ノードが並行して自動起動しています。 4-6. Slurm QoS による利用時間・優先順位の制御（TC-06 / Q4） 「学生と教員で最大実行時間やキューの優先順位を分けたい」という要件（Q4）は、Slurm アカウンティング（slurmdbd / sacctmgr）の QoS（Quality of Service） で制御します。本検証では以下の 2 つの QoS を定義します。 | QoS 名 | 想定ロール | 最大実行時間 (MaxWall) | ジョブ優先度 (Priority) | リソース上限 (MaxTRESPerUser) | | --- | --- | --- | --- | --- | | normal | 一般・学生 (user01) | 01:00:00（1 時間） | 10 | cpu=4（または gres/gpu=1） | | high | 教員・優先研究 (user02) | 24:00:00（24 時間） | 1000 | cpu=16（または gres/gpu=4） | <details><summary>06-configure-slurm-qos.sh: slurmdbd（MariaDB）の有効化と sacctmgr による QoS 作成（クリックで展開）</summary> # 06-configure-slurm-qos.sh（ヘッドノードの root で実行） # 0. MariaDB と slurmdbd の構成・起動（Job Accounting 未有効時） dnf install -y mariadb-server && systemctl enable --now mariadb mysql -e "CREATE DATABASE IF NOT EXISTS slurm_acct_db; CREATE USER IF NOT EXISTS 'slurm'@'localhost' IDENTIFIED BY 'slurm123'; GRANT ALL ON slurm_acct_db.* TO 'slurm'@'localhost'; FLUSH PRIVILEGES;" cat << 'EOF' > /sched/slurm-poc01/slurmdbd.conf AuthType=auth/munge DbdHost=localhost StorageType=accounting_storage/mysql StorageHost=localhost StorageUser=slurm StoragePass=slurm123 StorageLoc=slurm_acct_db LogFile=/var/log/slurm/slurmdbd.log PidFile=/var/run/slurmdbd.pid SlurmUser=slurm EOF chown slurm:slurm /sched/slurm-poc01/slurmdbd.conf && chmod 600 /sched/slurm-poc01/slurmdbd.conf systemctl enable --now slurmdbd cat << 'EOF' > /sched/slurm-poc01/accounting.conf AccountingStorageType=accounting_storage/slurmdbd AccountingStorageHost=localhost AccountingStorageEnforce=associations,limits,qos EOF systemctl restart slurmctld # 1. normal と high の 2 つの QoS を作成 sacctmgr -i add qos normal set MaxWall=01:00:00 Priority=10 MaxTRESPerUser=cpu=4 sacctmgr -i add qos high set MaxWall=24:00:00 Priority=1000 MaxTRESPerUser=cpu=16 # 2. クラスターアカウントにユーザーを登録し、既定 QoS を割り当て sacctmgr -i add user user01 Account=root DefaultQOS=normal QOS=normal sacctmgr -i add user user02 Account=root DefaultQOS=high QOS=high # 3. PriorityWeightQOS を有効化して設定反映 echo -e "\nPriorityType=priority/multifactor\nPriorityWeightQOS=10000" >> /sched/slurm-poc01/site_specific.conf scontrol reconfigure sacctmgr show qos format=Name,Priority,MaxWall,MaxTRESPerUser </details> 現在のユーザーと QoS の割り当てを確認する Linux に存在するユーザー、Slurm 管理 DB に登録されたユーザー、個々のジョブが使う QoS は、それぞれ分けて確認します。 以下は scheduler 上で実行する参照専用のコマンドです。全ユーザーの設定・履歴を見るため、管理者として sudo を使用しています。 <details><summary>ユーザー一覧・QoS 割り当て・ジョブで使用した QoS を確認するコマンド（クリックで展開）</summary> # Linux ユーザー：ユーザー名、UID、ホーム、ログインシェル（システム用も含む） getent passwd | cut -d: -f1,3,6,7 # Slurm 管理 DB に登録されたユーザー sudo sacctmgr show user format=User,DefaultAccount,AdminLevel # クラスタ・アカウント・パーティションごとの利用可能 QoS と既定 QoS sudo sacctmgr show assoc format=Cluster,Account,Partition,User,QOS%40,DefaultQOS # 各 QoS の具体的な制限・優先度 sudo sacctmgr show qos format=Name,Priority,MaxWall,MaxTRESPerUser # 今日の全ユーザーのジョブで実際に使われた QoS（実行中・終了済み） sudo sacct -a -S today -X --format=JobID,User,JobName,QOS,State,Elapsed </details> | 表示項目 | 意味 | | --- | --- | | QOS | そのユーザーの関連付け（association）で利用できる QoS | | DefaultQOS | ジョブ投入時に --qos を指定しなかった場合の既定 QoS | | sacct の QOS | 個々のジョブが実際に使用した QoS | | AdminLevel | Slurm アカウンティングの管理権限。high / normal の区分とは別 | この記事の設定手順では、user01 は利用可能・既定ともに normal、user02 は利用可能・既定ともに high です。azureuser の QoS は上のユーザー追加コマンドでは指定していないため、実際の関連付けを確認してください。show assoc で User が空欄の行はアカウント単位の設定で、別のユーザーを表しているわけではありません。 Linux にユーザーが存在していても Slurm 管理 DB に登録されているとは限りません。また、同一ユーザーでもクラスタ・アカウント・パーティションによって関連付けが異なる場合があります。設定例ではなく、現在のコマンド出力を実設定として確認してください。 > 出典: Slurm sacctmgr（ユーザー・関連付け・QoS の管理） / Slurm sacct（ジョブ履歴） !sacctmgr での QoS 設定と user01 の上限時間超過ジョブが QOSMaxWallDurationPerJobLimit でブロックされる画面 QoS の検証結果。normal QoS（上限 1 時間）が割り当てられた user01 が --time=02:00:00（2 時間）のジョブ（JOBID 6）を投入すると、REASON が (QOSMaxWallDurationPerJobLimit) となって実行前にブロックされます。 設定後の検証動作: 1. 上限時間を超えるジョブのブロック（保留）と対処法: user01（normal QoS：上限 1 時間）が sbatch --time=02:00:00 gpu-job01.sh を投入すると、REASON が (QOSMaxWallDurationPerJobLimit) となって実行前にブロックされます。 - ブロックされたジョブはどうなるか: 自動的には削除（キャンセル）されず、キュー内で PD（Pending：保留）状態のままホールドされます。この状態では計算ノードの自動起動（オートスケール）も一切トリガーされないため、無駄な VM 課金が発生することはありません。 - 解除・対処する方法: - 利用者本人が scancel <JOBID> で一度取り消し、--time=01:00:00 以下に修正して再投入する（または scontrol update JobId=<JOBID> TimeLimit=01:00:00 で要求時間を短縮するとそのまま実行待ちに進む）。 - 管理者が特例として長時間実行を許可する場合は、sudo scontrol update JobId=<JOBID> QOS=high でジョブの QoS を変更するとブロックが解除される。 - ※もしキューに PD で残すのではなく、sbatch を叩いた瞬間にエラーで弾きたい（投入自体を拒否したい）場合は、パーティション設定や slurm.conf の EnforcePartLimits=ALL を組み合わせる運用も可能です。 2. 待機ジョブの優先順位制御: PriorityWeightQOS=10000 により、sprio で normal QoS と high QoS の優先度スコアに明確な差がつき、空きノードが high QoS のジョブへ優先割り当てされます。 3. ユーザー別の利用実績集計: sudo sacct -a -X --format=JobID,User,Partition,QOS,Elapsed,State を実行することで、誰がどの QoS で何時間クラスタを使用したかを後から正確に集計できます。 4-7. Azure NetApp Files のユーザークォータによる容量制限（TC-07 / Q5） 「特定のユーザーが巨大なデータセットやチェックポイントを置いて共有ストレージをパンクさせるのを防ぎたい」という要件（Q5）は、Azure NetApp Files（ANF）の個人ユーザークォータ（Individual User Quota） で制御します。 <details><summary>07-set-anf-user-quota.sh: ANF ボリュームに対する UID 単位の容量クォータ設定（クリックで展開）</summary> # 07-set-anf-user-quota.sh # user01 (UID: 20003) に対して 1 GiB (1048576 KiB) のユーザークォータを設定 az netappfiles volume quota-rule create \ -g "$RG" --account-name "$ANF_ACCT" --pool-name "$ANF_POOL" --volume-name "$ANF_VOL" \ -n "quota-user01" -l "$LOCATION" \ --quota-type IndividualUserQuota --quota-target "20003" --quota-size 1048576 </details> !Azure Portal の Azure NetApp Files ボリューム（vol01）における個人ユーザークォータルール（1 GiB）の設定画面 Azure Portal 上の vol01 > [クォータ ルール] 画面。UID 20003（user01）に対して 1 GiB (1,048,576 KiB) の個別ユーザークォータ（IndividualUserQuota）が適用されています。 クォータ反映後、Linux クライアント（scheduler）上で quota -s コマンドを実行すると、ANF ボリューム（10.0.253.4:/vol01）側の rquotad から user01 の上限（1024M）と現在の消費量がリアルタイムに取得できることを確認できます。 <details><summary>quota -s によるユーザークォータ上限と消費量追跡の確認コマンド（クリックで展開）</summary> # user01 (UID: 20003) のクォータ上限（1024M）と 800MB ファイル作成時の消費量追跡を確認 sudo -u user01 quota -s sudo -u user01 dd if=/dev/zero of=/shared/home/user01/test_800m.bin bs=1M count=800 sudo -u user01 quota -s </details> !Linux クライアント上で quota -s コマンドにより ANF の 1024M ユーザークォータ上限を確認し dd で書き込みテストを実行している画面 sudo -u user01 quota -s の実行画面。10.0.253.4:/vol01 の quota / limit が 1024M（1 GiB）として認識されており、800 MB ファイル書き込み（dd）後の再実行では space が 16K から 801M へ増加して ANF 側で UID 単位の消費量が追跡されます。 このように、「計算リソース・ジョブ実行時間のガバナンスは Slurm の QoS」「ホーム領域の容量ガバナンスは ANF のユーザークォータ」 と役割分担することで、OS 側の複雑なクォータ設定を自前で管理することなく安全に共用環境を運用できます。 > 出典: Azure NetApp Files ボリュームの既定および個別のユーザーとグループのクォータを管理する 4-8. GPU SKU の置き換え・複数世代 GPU の混在運用（TC-08 / Q7） 「当初は入手しやすい GPU SKU で始め、後から H100（Standard_NC40ads_H100_v5 / Standard_ND96isr_H100_v5）や A100（Standard_NC24ads_A100_v4）に変更・追加したい」という要件（Q7）に対しては、CycleCloud のクラスタ設定で以下の 2 通りのアプローチが取れます。 !CycleCloud の VM サイズ選択画面（Compute Type: GPU）における各種 GPU SKU と単価・クォータ・RDMA 対応の一覧 CycleCloud の Select a machine type ダイアログで Compute Type: GPU を選択した画面。A100（Standard_NC24ads_A100_v4）や RTX PRO 6000、NVv4 などの各 GPU SKU について、コア数・メモリ・時間単価・クォータ状況・SR-IOV / RDMA 対応可否を一覧比較して選択できます。 1. 既存パーティションの VM SKU 差し替え: CycleCloud GUI のクラスタ編集画面（Required Settings → HPC VM Type または GPU VM Type）で VM サイズを変更して Save し、ヘッドノードで sudo azslurm scale を実行するだけで、次回起動する計算ノードから新しい GPU SKU に切り替わります。 2. GPU 種類ごとのパーティション分割（推奨）: CycleCloud 8 の Slurm テンプレートには最初から hpc・gpu・htc・dynamic の複数パーティションが用意されています。日常のコードデバッグ・前処理用には汎用 VM や低価格 GPU パーティションを割り当て、本番の大規模学習用には A100 / H100 パーティションを割り当てることで、ユーザーは sbatch の --partition 指定を変えるだけで同じ /shared/home/$USER 環境のまま複数の GPU 世代を使い分けられます。 --- 5. ハマったポイント・運用上の注意点 今回の実機構築・検証を通じて実際に遭遇したハマりどころと解決策をまとめます。 | # | ハマりどころ・事象 | 原因 | 対処法 | | --- | --- | --- | --- | | 1 | CycleCloud GUI の Subscriptions > Create が Loading... のまま進まない | 2026年3月31日以降、新規サブネットが既定でプライベートサブネット（defaultOutboundAccess: false）になり、パブリック IP なしの cc-server01 から management.azure.com へ通信できなくなったため | クラスタ用サブネット（subnet-cluster01）に Azure NAT Gateway をアタッチして ARM API 等へのアウトバウンド通信を明示的に許可する | | 2 | サブスクリプション登録直後に VM サイズ選択（Choose）を開くと No rows in the current filter で空になる | サブスクリプション追加直後の約 1〜2 分間は、CycleCloud がバックグラウンドで Azure の全 VM SKU（約 7 万件）と価格情報を DB に同期しているため | cycle_server.log に Stored 71,431 records of type Azure.MachineType が出力されるまで 1〜2 分待ってからダイアログを開き直す（クォータ 0 の SKU も見たい場合は Hide Unavailable のチェックを外す） | | 3 | Network Attached Storage タブで /shared に ANF を設定する際、内蔵 NFS と競合する・またはマウントに失敗する | ① File-system Mount for /shared の Use Builtin NFS にチェックが残っていると内蔵ディスクと競合する<br>② /shared 専用欄の 3 行目は Mount Point ではなく Export Path、4 行目は Mount Options になっている | File-system Mount for /shared の Use Builtin NFS のチェックを外し、IP Address: 10.0.253.4、Export Path: /vol01、Mount Options: 空欄 に設定する（下部の Additional Filesystem Mount は使わない） | | 4 | useradd -u 20001 user01 を実行すると UID 20001 is not unique エラーになる | CycleCloud が作成するクラスタ管理者ユーザー（azureuser）に最初から UID 20001 が割り当てられているため | 追加の一般ユーザーは 20002 以降の UID を割り当てる（ANF ユーザークォータの --quota-target も実際の UID に合わせる） | | 5 | sacctmgr で QoS を設定しても実行時間制限や sprio の優先度差が効かない | 既定では accounting.conf が accounting_storage/none になっており、また site_specific.conf に PriorityWeightQOS が設定されていないため | slurmdbd（MariaDB）を構成して AccountingStorageEnforce=associations,limits,qos を有効化し、site_specific.conf（末尾改行に注意）へ PriorityType=priority/multifactor と PriorityWeightQOS=10000 を追記して scontrol reconfigure を実行する | | 6 | NFSv3（hard マウント）上の ANF でバッファード dd によりクォータ上限を超過させるとプロセスが D 状態でハングする | AlmaLinux 8.10 カーネルのページキャッシュ書き戻し（nfs_file_flush）で NFS3ERR_DQUOT が返った際、hard マウントのリトライ待ちに入り /shared/home 全体の I/O が詰まることがある | クォータの動作確認には quota -s コマンドを使用する。書き込み上限テストを行う場合は oflag=direct や小さいブロックサイズを使い、万一 .nfs* ロックが残った場合は一時的にクォータ上限を広げて削除する | | 7 | GPU ジョブ（--gres=gpu:1）投入時に Requested node configuration is not available となる | CycleCloud の autoscale.json が対象 GPU VM SKU の GPU 枚数（slurm_gpus）を自動解決できていないケースがある | ヘッドノードの /opt/azurehpc/slurm/autoscale.json の default_resources に対象 VM サイズと slurm_gpus 枚数を明記し、azslurm scale と azslurm buckets で反映を確認する | | 8 | 検証終了後に課金が継続してしまう | 計算ノードが 0 台でも、CycleCloud サーバー・Slurm ヘッドノード・Windows VDI・Azure Bastion・NAT Gateway・ANF プール（4 TiB）は起動中課金が発生する | デモ・検証期間外は azslurm keep_alive --remove で常時起動を解除し、VM 停止（Deallocate）、および長期中断時は Bastion と ANF ボリューム/プールの削除を実施する | <details><summary>参考: GPU枚数がSlurmに反映されない場合の autoscale.json 修正手順（クリックで展開）</summary> # Slurm ヘッドノードで root になり autoscale.json をバックアップ・編集 sudo -i cd /opt/azurehpc/slurm cp -p autoscale.json autoscale.json.org # default_resources セクションに使用する GPU VM SKU と GPU 枚数を追記した上でスケール設定を再生成 azslurm scale azslurm buckets </details> > 出典: GitHub - Azure/cyclecloud-slurm: Incorrect number of GPUs --- まとめ - Azure CycleCloud 8 + Slurm + Azure NetApp Files を組み合わせることで、大学・研究機関で求められる「バッチ処理」「JupyterLab による対話操作」「複数ユーザー共用」「QoS による時間・優先度制限」「ANF ユーザークォータによる容量制限」「0 台からのオートスケールと常時起動の併用」を 1 つの閉域クラスタで実現できる。 - Azure Bastion + Windows VDI を組み合わせることで、クラスタ内の全 VM からパブリック IP を排除しつつ、CycleCloud 管理画面や計算ノード上の JupyterLab へ安全にブラウザアクセスできる。 - AlmaLinux 8.10 HPC イメージを採用することで GPU ドライバー導入のオーバーヘッドをなくし、将来の GPU SKU 変更や複数世代 GPU パーティションの混在にも柔軟に対応できる。 参考リンク - Azure CycleCloud のドキュメント (CycleCloud 8) - Azure CycleCloud での Slurm の使用 - クラスタの自動スケーリングを構成する (Azure CycleCloud) - Azure HPC/AI VM イメージ - Azure での既定のアウトバウンド アクセス (プライベート サブネット既定化) - Azure NetApp Files の既定および個別のユーザーとグループのクォータについて - Azure NetApp Files ボリュームの既定および個別のユーザーとグループのクォータを管理する - GitHub - Azure/cyclecloud-slurm > 本記事は GitHub Copilot および Microsoft Foundry を活用して作成されています。内容の正確性については各公式ドキュメントをご確認ください。
ARM template to architecture diagram from a GitHub URL --- title: "ARM template to architecture diagram from a GitHub URL" published: true description: "ARM template to architecture diagram: change one word in a GitHub template URL to get architecture and dependency views, with no Azure credentials needed." tags: azure, architecture, devops, github cover_image: dev-to-uploads.s3.us-east-2.amazonaws.com canonical_url: cloudeval.ai --- > TL;DR: To turn a GitHub-hosted ARM template into an architecture diagram, keep the URL path and swap github.com for cloudeval.ai. Cloudeval reads the ARM JSON, looks for a matching parameters file, creates a project and opens architecture and dependency views. No Azure credentials are needed. Compile Bicep to ARM JSON first. By Prateek Singh, founder of Ganak AI Labs. Updated 6 October 2026. Someone drops a link in the channel: "Can we use this Quick Start template for the new environment?" It points to an azuredeploy.json on GitHub. You open it and get about 250 lines of nested JSON. Variables build resource IDs, dependsOn arrays pile up, and a public IP is easy to miss. Before anyone deploys, you need to know what the template builds and how it's wired. A diagram answers that faster than a read-through. Here's the shortest path from a GitHub URL to one: a one-word change in the address bar. Disclosure: I'm the founder of Cloudeval. Key terms used below - ARM template: an Azure Resource Manager (ARM) JSON file that declares the resources Azure deploys. - Azure Quick Start Templates: Microsoft's public sample repository of ARM templates, Azure/azure-quickstart-templates on GitHub. - Architecture view and dependency view: two Cloudeval diagram layouts. The first shows what exists. The second shows what depends on what. Why is an ARM template hard to review as raw JSON? ARM JSON is written for Resource Manager, not for people. The architecture is spread across resources, parameters, variables, nested deployments and implicit relationships. You only see the shape of the system once you've resolved all of that in your head. A diagram gives reviewers a shared picture first. It should tell you: - what resources the template creates - which services depend on each other - whether the network shape matches what you expected - whether public-facing resources stand out - whether the template should be split up How do I turn a GitHub ARM template URL into a diagram? 1. Pick a public or accessible ARM JSON template on GitHub. 2. Swap github.com for cloudeval.ai in the URL and press Enter. 3. Check the detected parameters file, then choose Review & Create. 4. Read the architecture view, then the dependency view. 5. Run reports, or script the same flow with the CLI. Step 1: Pick a GitHub-hosted ARM JSON template The shortcut works on ARM JSON that Cloudeval can read from GitHub. blob, tree and raw GitHub URLs all work when they point to a supported template (GitHub URL guide). This walkthrough uses a Quick Start that deploys one VM with two NICs in a virtual network: https://github.com/Azure/azure-quickstart-templates/blob/master/quickstarts/microsoft.compute/1vm-2nics-2subnets-1vnet/azuredeploy.json It creates a virtual machine, a storage account, two network interfaces, a public IP address, a virtual network and two network security groups. That's small enough to check against the diagram by eye. Working in Bicep? Compile it first with az bicep build --file main.bicep and use the generated ARM JSON. Cloudeval reads Bicep compiled to ARM JSON, not .bicep files. Step 2: Swap github.com for cloudeval.ai Keep everything after the host exactly as it is. Change only the domain: https://cloudeval.ai/Azure/azure-quickstart-templates/blob/master/quickstarts/microsoft.compute/1vm-2nics-2subnets-1vnet/azuredeploy.json Press Enter. Cloudeval opens Quick Project Creation with the GitHub URL already filled in. You need to be signed in to Cloudeval. You don't need an Azure subscription. !Cloudeval Quick Project Creation dialog importing an Azure Quick Start ARM template after github.com was changed to cloudeval.ai Quick Project Creation picks up the URL and starts the import. Step 3: Check the parameters file and create the project Many templates need a parameters file. When the URL follows a common naming pattern, Cloudeval looks for a matching one in the same repository. You'll see Looking..., Found or Not found beside the parameters field. If it picks the wrong file, paste the right parameters URL. Confirm the project name, provider and URLs, then choose Review & Create. Cloudeval creates the project and opens the workspace. If setup is still running, let it finish before you run reports or share the project. Step 4: Read the architecture view, then the dependency view Start with the architecture view. It answers "what exists?": resource groups, major resources, service boundaries and topology. Check it against the template. For this Quick Start, you should find the VM, both NICs, the public IP and the two network security groups. !Cloudeval architecture diagram of an Azure template with a network interface selected and its problems panel listing a public IP attached Select a resource to see its properties and findings beside the graph. Switch to the dependency view when relationships matter. It shows upstream and downstream resources, paths that widen the blast radius, and where the declared topology differs from what you expected. The two views answer different questions, so use both. Here's a 30-second look at how the diagrams are laid out: {% embed youtube.com %} Step 5: Run reports, or script the same flow The URL only creates the project. Run reports from the workspace once import is done. The Architecture & Dependencies tab ties each finding to its dependency context and affected resources. !Cloudeval Architecture and Dependencies report tab for an Azure project showing nodes, mapped findings, critical paths and an architecture evidence map Findings mapped to dependency context and affected resources. Prefer the terminal? The CLI takes the same GitHub URL: npm install -g @ganakailabs/cloudeval-cli # Node.js 20+ cloudeval login cloudeval projects create \ --template-url https://github.com/Azure/azure-quickstart-templates/blob/master/quickstarts/microsoft.compute/1vm-2nics-2subnets-1vnet/azuredeploy.json \ --name "Quick Start: 1 VM, 2 NICs" \ --provider azure \ --format json \ --output ./cloudeval-project.json PROJECT_ID=$(jq -r '.data.project.id' ./cloudeval-project.json) cloudeval projects export-diagram "$PROJECT_ID" \ --layout dependency --format svg --output dependency.svg --non-interactive export-diagram takes --layout architecture|dependency and --format png|jpeg|svg. That's handy for a design doc or a PR description. When should you use repository sync or upload instead? The URL swap is built for speed on a single template. Pick another path when the source is more complicated: | Situation | Use | Why | |---|---|---| | One public ARM JSON file | GitHub URL swap | Fastest browser path, with parameters detection | | Linked or nested templates (templateLink) | Workspace import with .cloudeval/config.yaml | Resolves relative linked files from an entry template | | Private repo, branch history, refresh on push | GitHub repository sync (Cloudeval GitHub App) | Keeps commit and branch context on the project | | Template built locally or not reachable by URL | Local upload | Nothing has to be public | All the options are in Import an ARM or Bicep template. Is a generated diagram enough to approve a template? No. A diagram shows what the template declares. It doesn't prove what happens at deployment or at runtime. Microsoft's own tools still matter. The Bicep extension for VS Code has a built-in visualizer for a quick look while editing. az deployment group what-if is still the real answer to "what will change in this subscription?" What the URL flow adds is speed and a shareable review for templates you didn't write. There's no download and no Azure access, and the same project holds reports, findings and exports. Findings draw on 1,650+ checks. Not every project runs every check, and unsupported areas show as "Not assessed" instead of passing. It doesn't deploy anything, read your live subscription (that's a separate Cloud sync) or test routing and access in a deployed environment. Frequently asked questions How do I convert an ARM template to an architecture diagram? Import the ARM JSON into a Cloudeval Azure project, by upload or by GitHub URL, and open the architecture view. For a GitHub-hosted template, the quickest way is to replace github.com with cloudeval.ai in the URL and press Enter. Is there an Azure ARM template visualizer that works from a URL? Yes. Cloudeval accepts blob, tree and raw GitHub URLs that point to supported ARM JSON, and the cloudeval.ai host swap opens the import directly. For Bicep files open in VS Code, Microsoft's Bicep visualizer is the built-in option. Do I need a live Azure connection? No. You need a Cloudeval account to create the project, but ARM template import doesn't need Azure credentials. Live Cloud sync is a separate workflow. Does importing the template deploy Azure resources? No. Importing creates a Cloudeval project so you can review the template. Deploying Azure resources is a separate step that you run yourself. Can I use Bicep directly? Not in this workflow. Compile Bicep to ARM JSON first with az bicep build, then import the generated file with any parameters and linked files the project needs. Takeaway When someone shares an ARM template on GitHub, change one word in the URL and look at the dependency view before you read the JSON. It gives the team a picture to argue about. Then run what-if before you deploy. For the full reference answer, see How do I convert an ARM template to an architecture diagram? Related on cloudeval.ai - How do I convert Bicep to a cloud architecture diagram? - How do I generate an Azure architecture diagram automatically? - Azure architecture assessment before changes ship - What is IaC review? > Try it > > Take any ARM template URL on GitHub, swap github.com for cloudeval.ai, and open the dependency view. Start in the Cloudeval app and follow the GitHub URL guide. > About the author > > Prateek Singh is the founder of Ganak AI Labs and builds Cloudeval AI. > > cloudeval.ai | Docs | YouTube | GitHub | LinkedIn
Azure CLI still trusts compromised eMudhra CA; fix targeted for November 16
久し振りに Durable Functions に貢献した github.com/Azure/azure-functions-durable-extensio…
あまり具体的な情報があるわけではないけど、確認不可能な App Service の Release note を公開してくれているのはありがたい github.com/Azure/AppService/releases/tag/2026-10-…
Microsoft Foundry の評価で、正答と誤答をどう採点するか試してみた 執筆日 2026/10/22 参考資料 Microsoft Foundry でのクラウド評価の概要 データセットを使ってクラウド評価を実行する 評価結果を取得する 評価に必要なアクセス許可 評価のリージョン、制限、仮想ネットワーク azure-ai-evaluation の F1 スコア実装（GitHub） 検証内容 質問、回答、正解を手書きしたデータセットを登録し、その回答をそのまま採点しました。3件のうち1件はわざと誤答にしてあります。各エバリュエーターが正答と誤答をどう採点するか、行ごとのスコア、合否、理由を確認しました。 検証環境 モデル: gpt-6.1-sol（Global Standard） SDK: azure-ai-projects 2.8.0, openai 3.29.0 Python: 3.13 使うエバリュエーター 名前 種類 入力 何を測るか coherence LLM ジャッジ（1〜5、しきい値 3） query, response 文として筋が通っているか。事実の誤りは見ない f1 計算式（0〜1、しきい値 0.5） response, ground_truth 正解と単語がどれだけ重なっているか similarity LLM ジャッジ（1〜5、しきい値 3） query, response, ground_truth 正解と意味が一致しているか 今回は3つですが、他にもエバリュエーターはあります。 learn.microsoft.com 事前準備 ツールのインストール mkdir foundry-eval-handson && cd foundry-eval-handson python -m venv .venv source .venv/bin/activate # Windows は .venv\Scripts\activate pip install "azure-ai-projects>=2.8.0" azure-identity openai python-dotenv サインインと .env の作成 az login -t <tenant ID> az account show --query "{subscription:name, tenant:tenantId}" -o table 表示されたサブスクリプションとテナントが、Foundry プロジェクトのあるものと一致することを確認してください。 プロジェクト エンドポイントは、ポータルのプロジェクト概要ページで確認できます。AZURE_AI_MODEL_DEPLOYMENT_NAME には、モデル名ではなくデプロイ名を書きます。 .env AZURE_AI_PROJECT_ENDPOINT=https://<account_name>.services.ai.azure.com/api/projects/<project_name> AZURE_AI_MODEL_DEPLOYMENT_NAME=gpt-6.1-sol コード 手書きのテストデータもコードの中に持たせ、実行時に manual_data.jsonl へ書き出してからアップロードします。 evaluate_stored.py import json import os import time from datetime import datetime from dotenv import load_dotenv from azure.identity import DefaultAzureCredential from azure.ai.projects import AIProjectClient from azure.ai.projects.models import TestingCriterionAzureAIEvaluator from openai.types.eval_create_params import DataSourceConfigCustom from openai.types.evals.create_eval_jsonl_run_data_source_param import ( CreateEvalJSONLRunDataSourceParam, SourceFileID, ) load_dotenv() ENDPOINT = os.environ["AZURE_AI_PROJECT_ENDPOINT"] MODEL = os.environ["AZURE_AI_MODEL_DEPLOYMENT_NAME"] # 採点役のモデル project_client = AIProjectClient(endpoint=ENDPOINT, credential=DefaultAzureCredential()) openai_client = project_client.get_openai_client() 実行ごとに一意なバージョンを使い、同じ name / version の再登録を避ける RUN_VERSION = datetime.now().strftime("%Y%m%d%H%M%S") 手書きデータ。3 行目はわざと誤った回答にしている MANUAL_DATA_PATH = "manual_data.jsonl" MANUAL_ROWS = [ { "query": "Contoso の返金期限は購入から何日以内ですか？", "response": "購入から 30 日以内であれば全額返金されます。", "ground_truth": "購入から 30 日以内で、元の状態で返品された商品は全額返金されます。", }, { "query": "ダウンロード済みのデジタル商品は返金できますか？", "response": "ダウンロード後のデジタル商品は返金できません。", "ground_truth": "デジタル商品はダウンロード後は返金できません。", }, { "query": "30 日を過ぎた場合はどうなりますか？", "response": "いつでも現金で全額返金されます。", "ground_truth": "30 日を過ぎた場合は、サポートの判断でストアクレジットが発行されることがあります。", }, ] def build_testing_criteria() -> list: """エバリュエーターを定義する。response と ground_truth はデータセットの列を参照する""" return [ TestingCriterionAzureAIEvaluator( # LLM 採点: 文として筋が通っているか（1〜5） type="azure_ai_evaluator", name="coherence", evaluator_name="builtin.coherence", initialization_parameters={"model": MODEL}, data_mapping={"query": "{{item.query}}", "response": "{{item.response}}"}, ), TestingCriterionAzureAIEvaluator( # 計算式: 正解との単語の重なり（0〜1） type="azure_ai_evaluator", name="f1", evaluator_name="builtin.f1_score", data_mapping={ "response": "{{item.response}}", "ground_truth": "{{item.ground_truth}}", }, ), TestingCriterionAzureAIEvaluator( # LLM 採点: 正解と意味が一致しているか（1〜5） type="azure_ai_evaluator", name="similarity", evaluator_name="builtin.similarity", initialization_parameters={"model": MODEL}, data_mapping={ "query": "{{item.query}}", "response": "{{item.response}}", "ground_truth": "{{item.ground_truth}}", }, ), ] def wait_and_show(eval_object, eval_run): """評価実行の完了を待ち、集計と行ごとの結果を表示する""" while True: run = openai_client.evals.runs.retrieve(run_id=eval_run.id, eval_id=eval_object.id) if run.status in ("completed", "failed", "canceled"): break print(f" status={run.status} ... waiting") time.sleep(10) if run.status != "completed": raise RuntimeError(f"Evaluation run ended in {run.status}: {run.error}") run_data = run.to_dict(warnings=False) print("\n=== 集計 ===") print("result_counts:", run_data.get("result_counts")) for c in run_data.get("per_testing_criteria_results") or []: passed, failed, errored = c.get("passed", 0), c.get("failed", 0), c.get("errored", 0) judged = passed + failed pass_rate = f"{passed / judged:.2f}" if judged else "-" print( f" {c.get('testing_criteria')}: pass_rate={pass_rate} " f"(passed={passed}, failed={failed}, errored={errored})" ) print("\n=== 行ごとの結果 ===") items = list(openai_client.evals.runs.output_items.list(run_id=run.id, eval_id=eval_object.id)) for i, item in enumerate(items, 1): item_data = item.to_dict(warnings=False) query = (item_data.get("datasource_item") or {}).get("query") print(f"--- item {i}: {query}") for r in item_data.get("results") or []: if r.get("status") == "error": error = (r.get("sample") or {}).get("error") or {} print(f" {r.get('name')}: ERROR {error.get('message')}") continue print( f" {r.get('name')}: score={r.get('score')} " f"label={r.get('label')} threshold={r.get('threshold')}" ) if r.get("reason"): print(f" reason: {r.get('reason')}") print("\nReport URL:", run.report_url) return run def main(): print(f"model: {MODEL}") print("\n##### 格納済み応答の評価 #####") with open(MANUAL_DATA_PATH, "w", encoding="utf-8") as f: for row in MANUAL_ROWS: f.write(json.dumps(row, ensure_ascii=False) + "\n") dataset = project_client.datasets.upload_file( name="handson-manual-data", version=RUN_VERSION, file_path=MANUAL_DATA_PATH, ) print(f"dataset: {dataset.name} v{dataset.version} (id: {dataset.id})") eval_object = openai_client.evals.create( name="handson-dataset-evaluation", data_source_config=DataSourceConfigCustom( type="custom", item_schema={ "type": "object", "properties": { "query": {"type": "string"}, "response": {"type": "string"}, "ground_truth": {"type": "string"}, }, "required": ["query", "response", "ground_truth"], }, ), testing_criteria=build_testing_criteria(), ) eval_run = openai_client.evals.runs.create( eval_id=eval_object.id, name="manual-data-run", data_source=CreateEvalJSONLRunDataSourceParam( type="jsonl", source=SourceFileID(type="file_id", id=dataset.id), ), ) wait_and_show(eval_object, eval_run) if name == "main": main() 実行結果 出力（接続情報はマスク済み）model: gpt-6.1-sol 格納済み応答の評価 ##### dataset: handson-manual-data v20261021091729 (id: azureai://accounts/<account_name>/projects/<project_name>/data/handson-manual-data/versions/20261021091729) status=in_progress ... waiting status=in_progress ... waiting status=in_progress ... waiting === 集計 === result_counts: {'errored': 0, 'failed': 2, 'passed': 1, 'total': 3, 'skipped': 0} coherence: pass_rate=1.00 (passed=3, failed=0, errored=0) f1: pass_rate=0.33 (passed=1, failed=2, errored=0) similarity: pass_rate=0.67 (passed=2, failed=1, errored=0) === 行ごとの結果 === --- item 1: Contoso の返金期限は購入から何日以内ですか？ coherence: score=5.0 label=pass threshold=3 reason: Let's think step by step: 質問は購入から何日以内が返金期限かを尋ねており、回答は「購入から30日以内」と期限を明確に示しています。全額返金という補足も同じ条件に自然につながっており、簡潔で論理的にまとまった、理解しやすい回答です。 f1: score=0.6666666666666666 label=pass threshold=0.5 similarity: score=4.0 label=pass threshold=3 reason: 予測回答は「購入から30日以内」という返金期限と全額返金の点で正解と一致していますが、「元の状態で返品された商品」という条件が省略されています。 --- item 2: ダウンロード済みのデジタル商品は返金できますか？ coherence: score=5.0 label=pass threshold=3 reason: Let's think step by step: 質問はダウンロード済みのデジタル商品の返金可否を尋ねており、回答は返金できないと直接かつ明確に答えています。一文で完結していて論理的な混乱や不要な情報がなく、質問と回答のつながりも明瞭です。この簡潔な質問に対して必要な内容を過不足なく示しているため、非常に高い一貫性があります。 f1: score=0.0 label=fail threshold=0.5 similarity: score=5.0 label=pass threshold=3 --- item 3: 30 日を過ぎた場合はどうなりますか？ coherence: score=4.0 label=pass threshold=3 reason: Let's think step by step: The query asks what happens after 30 days, and the response directly states that a full cash refund is available at any time, implicitly including after 30 days. The sentence is clear, complete, and easy to understand, with no internal contradictions or disjointed ideas. It is coherent, though it does not elaborate on how the 30-day threshold affects the refund policy. f1: score=0.0 label=fail threshold=0.5 similarity: score=1.0 label=fail threshold=3 reason: 正解は「30日を過ぎた場合、サポートの判断でストアクレジットが発行されることがある」としているのに対し、予測回答は「いつでも現金で全額返金される」と述べており、返金条件・返金方法・確実性がすべて異なるため、同等性はほぼありません。 Report URL: ai.azure.com<masked>/build/evaluations/<eval_id>/run/<run_id> 結果のまとめ 行 内容 coherence f1 similarity 行の判定 1 正答 5 pass 0.67 pass 4 pass pass 2 正答 5 pass 0 fail 5 pass fail 3 誤答 4 pass 0 fail 1 fail fail スコアと合否（label）の関係 各エバリュエーターは、それぞれの尺度でスコアを出します（品質系は5段階、F1 は 0〜1 など）。そのスコアを既定のしきい値で pass / fail に変換します。coherence と similarity の既定しきい値は 3、F1 は 0.5 です。しきい値は SDK で上書きできます。 learn.microsoft.com ポータルでの確認 評価結果の画面 実行結果に表示された Report URL をブラウザーで開く 実行の一覧から manual-data-run を開く 行ごとのスコア、合否、reason を確認する 確認すること スクリプトの出力と、ポータル上のスコア・合否が一致しているか 同じ評価定義の中で、複数の実行を並べて比較できるか データセットの画面 ポータルの [データ] タブを開く handson-manual-data を開き、行をプレビューする 確認すること 登録した3行が、列名（query / response / ground_truth）どおりに表示されているか 実行ごとにバージョンが分かれて保存されているか