> ## Documentation Index > Fetch the complete documentation index at: https://confidence.spotify.com/llms.txt > Use this file to discover all available pages before exploring further. # Run PowerAnalysis > Runs a power analysis. ## OpenAPI ````yaml /api/stats/api/power.openapi.json post /v2/stats:runPowerAnalysis openapi: 3.1.0 info: title: Confidence Power API version: 1.0.0 description: API documentation for power servers: [] security: [] paths: /v2/stats:runPowerAnalysis: post: summary: Run PowerAnalysis description: Runs a power analysis. operationId: PowerAnalysis.run parameters: [] requestBody: content: application/json: schema: type: object properties: plan: $ref: '#/components/schemas/confidence.stats.v2.AnalysisPlan' description: >- [AnalysisPlan](/api-reference/schemas/analysisplan): The description of which hypotheses that are going to be tested, which group comparisons that are included, and how the result should be evaluated. data: $ref: '#/components/schemas/confidence.stats.v2.PowerAnalysisData' description: >- [PowerAnalysisData](/api-reference/schemas/poweranalysisdata): Additional data for the power analysis, only required to be set if hypotheses are relative or if ci_width should be computed. required: - plan required: true responses: '200': description: OK content: application/json: schema: type: object properties: hypotheses: type: array items: type: object properties: id: type: string description: The id of the hypothesis that was given by the user. requiredSampleSize: type: integer description: >- The sample size needed for the hypothesis to reach the given power for all segments. segments: type: array items: type: object properties: dimensions: type: array items: type: object additionalProperties: type: string description: >- The set of dimensions and their values that represent the segment. requiredSampleSize: type: integer description: >- The sample size needed to reach the given power. ciWidthAbs: type: number description: >- The estimated width of the confidence interval. ciWidthRel: type: number description: >- The estimated width relative to the segment baseline if provided, otherwise nan. baselineMean: type: number description: The baseline mean used in the calculation baselineVariance: type: number description: The variance used in the calculation varianceReductionRate: type: number description: >- The variance reduction rate used in the calculation description: The sample size needed for each segment. adjustedAlpha: type: number description: >- The alpha (false positive rate) used for the analysis, after adjustment. For one-sided tests, this is the alpha used in the selected tail. adjustedBeta: type: number description: >- The beta (1 - power) used for the analysis, after adjustment. description: The sample size for each hypothesis. requiredSampleSize: type: integer description: >- The sample size needed to get the desired power for all hypotheses. annotations: type: array items: type: object properties: context: type: array items: type: object properties: key: type: string enum: - CONTEXT_UNSPECIFIED - CONTEXT_HYPOTHESIS - CONTEXT_SEGMENT - CONTEXT_GROUP - CONTEXT_COMPARISON description: Specific context location, like a hypothesis. value: type: string description: The id. description: >- The context that the annotation was surfaced in, e.g. which group, hypothesis or segment it occurred in. info: type: string description: General information, not treated as an error. warning: type: string description: Warning, something might be wrong. error: type: object properties: details: type: string description: Error message. category: type: string enum: - ERROR_CATEGORY_UNSPECIFIED - ERROR_CATEGORY_GENERAL - ERROR_CATEGORY_DATA_VALIDATION_ERROR - ERROR_CATEGORY_ANALYSIS_ERROR - ERROR_CATEGORY_NO_DATA description: General category of the error. description: Error, something has gone wrong. description: List of issues discovered during the power analysis. statsCacheEntry: type: string description: Cache entry used, empty if not cached. security: - admin: [] servers: - url: https://stats.eu.confidence.dev - url: https://stats.us.confidence.dev - url: https://stats.confidence.dev components: schemas: confidence.stats.v2.AnalysisPlan: title: AnalysisPlan type: object description: |- The description of which hypotheses that are going to be tested, which group comparisons that are included, and how the result should be evaluated. properties: alpha: type: number description: >- The desired overall false positive rate (multiple testing will be applied automatically, so you don't need to adjust this number). power: type: number description: >- The desired statistical power, will be adjusted according to the decision rule. groups: type: array items: type: object properties: id: type: string description: User-defined identifier of the group. weight: type: integer description: >- The weight of the group, i.e., the number of samples allocated to group i will approximately be `weight_i / sum(weight_i)`. required: - id - weight description: The list of groups/treatments that will be included. comparisons: type: object properties: oneVsAll: type: object properties: baseline: type: string description: The group that will act as the baseline. required: - baseline description: Compare all groups against one. allVsAll: type: object description: Compare all groups against each other. custom: type: object properties: pairs: type: array items: type: object properties: baseline: type: string description: The group that will act as the baseline. compared: type: string description: The group that will be compared to the baseline. required: - baseline - compared description: List of groups that will be compared. required: - pairs description: Compare groups according to the custom list of pairs. description: >- Description how groups will be compared to each other. Note that some analysis methods requires all groups, in those cases that will override this behaviour. hypotheses: type: array items: type: object properties: id: type: string description: >- A used-defined id of the hypothesis that will be present in the result. type: type: object properties: superiority: type: object properties: preferredDirection: type: string enum: - DIRECTION_UNSPECIFIED - INCREASE - DECREASE - ANY description: >- The direction in which you want to declare superiority. minimumDetectableEffect: type: number description: >- The minimum effect that can be detected with the given power level. Typically the size of the effect that is of practical significance. The scale of this is relative to the segment baseline mean and requires the baseline to be set to be able to do power analysis. required: - preferredDirection description: >- A superiority hypothesis tests whether an estimate in one group is different, greater or lesser than an estimate of another group. nonInferiority: type: object properties: preferredDirection: type: string enum: - DIRECTION_UNSPECIFIED - INCREASE - DECREASE description: >- The direction in which you allow increases larger than the margin of error. nonInferiorityMargin: type: number description: >- The margin of error when declaring non-inferiority. The scale of this is relative to the segment baseline mean and requires the baseline to be set to be able to do power analysis. required: - preferredDirection - nonInferiorityMargin description: >- A non-inferiority hypothesis tests whether an estimate in one group is greater or lesser, with some margin of error, than an estimate of another group. jointEquivalence: type: object properties: minimumDetectableDeviation: type: number description: >- The minimum detectable deviation in all of the groups, i.e., if you have three groups with a minimum detectable deviation of 0.1 that means that if only one group differs in 0.1 it should be detectable. required: - minimumDetectableDeviation description: >- A joint equivalence hypothesis tests whether multiple estimates are the same or equal to some specified value. description: >- Determines whether you are testing a superiority or non-inferiority test. dataType: type: string enum: - DATA_TYPE_UNSPECIFIED - DATA_TYPE_CONTINUOUS - DATA_TYPE_BINARY - DATA_TYPE_COUNT description: >- The type of the underlying data, some methods might be able to take better advantage of the data if this is set, by default it is assumed that the data is continuous. segments: type: array items: type: object properties: dimensions: type: array items: type: object additionalProperties: type: string description: >- The set of dimensions and their values that represent the segment. params: type: object properties: chi2: type: object properties: expectedFrequency: type: array items: type: object properties: group: type: string description: The user-defined id of the group. frequency: type: number description: >- The expected frequency compared to the total count of this group. required: - group - frequency description: The expected frequency of each group. required: - expectedFrequency description: Analysis parameters for the χ² test. zTest: type: object description: Analysis parameters for the z-test. gstZTest: type: object properties: maxSampleSize: type: integer description: >- An estimate of the maximum sample size for the segment. required: - maxSampleSize description: Analysis parameters for the sequential z-test. ratio: type: object description: Analysis parameters for the ratio test. tunncDesign: type: object properties: optimalSampleSize: type: number description: >- Parameter that determines at which sample size where the bounds for the confidence intervals will be as tight as possible. It must be a positive number. The method is not too sensitive to this number, but err on underestimating it. If not set, a default value of 1 will be used. weight: type: array items: type: object additionalProperties: type: number description: >- Optionally override the weight passed the non-parametric test, if not set the group weights will be used. description: >- Analysis parameters for the non-parametric sequential test. gstRatio: type: object properties: maxSampleSize: type: integer description: >- An estimate of the maximum sample size for the segment. required: - maxSampleSize description: Analysis parameters for the sequential ratio test. asympCs: type: object properties: optimalSampleSize: type: number description: >- Parameter that determines at which sample size where the bounds for the confidence intervals will be as tight as possible. It must be a positive number. The method is not too sensitive to this number, but err on underestimating it. If not set, a default value of 10 will be used. description: >- Analysis parameters for the non-parametric sequential test. asympCsRatio: type: object properties: optimalSampleSize: type: number description: >- Parameter that determines at which sample size where the bounds for the confidence intervals will be as tight as possible. It must be a positive number. The method is not too sensitive to this number, but err on underestimating it. If not set, a default value of 10 will be used. description: >- Analysis parameters for the non-parametric sequential ratio test. description: The method that will be used to test the hypothesis, required: - params description: >- List of segments to test. If you don't need segmentation, you can specify the overall segment with an empty map. required: - id - type - segments description: >- List of hypotheses that should be tested, typically these represents the change in one or more metrics. decisionRule: type: object properties: items: type: array items: type: object properties: hypothesis: type: string description: The id of a hypothesis. rule: type: object description: A nested decision rule. description: >- A possibly nested boolean expression, where each term is either the id of an hypothesis or a nested boolean expression. operator: type: string enum: - LOGICAL_OPERATOR_UNSPECIFIED - AND - OR description: A boolean operator AND or OR that occurs between the items. required: - items - operator description: >- A nested expression that decides when it is valid to "ship" the experiment. This expression will impact both how the multiple testing adjustment and power adjustment is performed, and determine the shipping recommendations. A common example is success and guardrail metrics, for example with two success metric hypotheses S1, S2 and two guardrails hypotheses G1, G2 the decision rule could be formed as (G1 AND G2) AND (S1 OR S2). Assuming that the guardrails are non-inferiority hypotheses and the success metrics are superiority hypotheses, this means that both guardrails must be significantly non-inferior and at least one success metric must be significant. If not set, then an OR rule will be assumed across all hypotheses. The logical operator that an hypotheses is part of, will also determine the logical operator between segments. Meaning that if an hypothesis is part of an AND expression, the rule will only evaluate to true if it is significant on all segments, and similar for an OR expression. settings: type: object properties: disableSegmentAlphaAdjustment: type: boolean description: >- If true, disables the Bonferroni correction for the number of segments. By default, alpha is adjusted for the total number of tests including all segments. Set to true if you want to treat segments as independent analyses without multiple testing correction across them. description: Settings that control how the analysis is performed. required: - alpha - power - groups - comparisons - hypotheses confidence.stats.v2.PowerAnalysisData: title: PowerAnalysisData type: object description: Data used for the power analysis properties: hypotheses: type: array items: type: object properties: id: type: string description: The id of the hypothesis. segments: type: array items: type: object properties: dimensions: type: array items: type: object additionalProperties: type: string description: >- The set of dimensions and their values that represent the segment. powerData: type: object properties: chi2: type: object description: Power data for the χ² test. zTest: type: object properties: baselineMean: type: number description: >- The mean of the baseline group, only needed if data is binary. baselineVariance: type: number description: The variance of the baseline group. adjustment: type: object properties: baselineVariance: type: number description: Variance of the covariate. baselineCovariance: type: number description: >- Covariance of the covariate with the variable used for the mean. required: - baselineVariance - baselineCovariance description: >- Summary data for power analysis with covariate adjustment. required: - baselineVariance description: Power data for the z-test. gstZTest: type: object properties: baselineMean: type: number description: >- The mean of the baseline group, only needed if data is binary. baselineVariance: type: number description: The variance of the baseline group. adjustment: type: object properties: baselineVariance: type: number description: Variance of the covariate. baselineCovariance: type: number description: >- Covariance of the covariate with the variable used for the mean. required: - baselineVariance - baselineCovariance description: >- Summary data for power analysis with covariate adjustment. required: - baselineVariance description: Power data for the sequential z-test. ratio: type: object properties: baselineNumeratorVariance: type: number description: >- The variance of the numerator of the baseline group. baselineDenominatorVariance: type: number description: >- The variance of the denominator of the baseline group. baselineCovariance: type: number description: >- The covariance of the numerator and denominator of the baseline group. baselineNumeratorMean: type: number description: The mean of the numerator in the baseline group baselineDenominatorMean: type: number description: >- The mean of the denominator in the baseline group required: - baselineNumeratorVariance - baselineDenominatorVariance - baselineCovariance - baselineNumeratorMean - baselineDenominatorMean description: Power data for the ratio test. tunncDesign: type: object description: Power data for the non-parametric sequential test. gstRatio: type: object properties: baselineNumeratorVariance: type: number description: >- The variance of the numerator of the baseline group. baselineDenominatorVariance: type: number description: >- The variance of the denominator of the baseline group. baselineCovariance: type: number description: >- The covariance of the numerator and denominator of the baseline group. baselineNumeratorMean: type: number description: The mean of the numerator in the baseline group baselineDenominatorMean: type: number description: >- The mean of the denominator in the baseline group required: - baselineNumeratorVariance - baselineDenominatorVariance - baselineCovariance - baselineNumeratorMean - baselineDenominatorMean description: Power data for the sequential ratio test. asympCs: type: object properties: baselineMean: type: number description: >- The mean of the baseline group, only needed if data is binary. baselineVariance: type: number description: The variance of the baseline group. adjustment: type: object properties: baselineVariance: type: number description: Variance of the covariate. baselineCovariance: type: number description: >- Covariance of the covariate with the variable used for the mean. required: - baselineVariance - baselineCovariance description: >- Summary data for power analysis with covariate adjustment. required: - baselineVariance description: Power data for the non-parametric sequential test. asympCsRatio: type: object properties: baselineNumeratorVariance: type: number description: >- The variance of the numerator of the baseline group. baselineDenominatorVariance: type: number description: >- The variance of the denominator of the baseline group. baselineCovariance: type: number description: >- The covariance of the numerator and denominator of the baseline group. baselineNumeratorMean: type: number description: The mean of the numerator in the baseline group baselineDenominatorMean: type: number description: >- The mean of the denominator in the baseline group required: - baselineNumeratorVariance - baselineDenominatorVariance - baselineCovariance - baselineNumeratorMean - baselineDenominatorMean description: >- Power data for the non-parametric sequential ratio test. description: Method-specific data for the power analysis. expectedSampleSize: type: integer description: >- Estimated sample size of the segment, if set ci_width will be provided. required: - powerData description: The data for each segment. required: - id - segments description: Data for each hypothesis. required: - hypotheses securitySchemes: admin: type: http scheme: bearer ````