Workflows — Fitting Models via model:

Overview

This workflow demonstrates fitting mathematical models to data using both traditional and flexible parameter mapping approaches. The system provides:

The processor consumes data from the context channel and writes fitted coefficients back into the context. It supports both traditional fixed parameter names (x_values, y_values) and flexible parameter mapping for complex data structures.

Key Features

Traditional Interface (Backward Compatible)

Uses standard x_values and y_values parameter names with configurable output keys. Maintains full compatibility with existing pipelines.

Flexible Parameter Mapping

Supports custom parameter names and nested path extraction using dot notation (e.g., "gaussian_fit_parameters.std_dev_x"). Automatically handles different data structure formats through dynamic processor creation.

Data Structure Compatibility

  • Single dictionaries: {"key": [value1, value2, ...]}

  • Lists of dictionaries: [{"key": value1}, {"key": value2}, ...] (common from slicers)

  • Nested structures: Multi-level path extraction with dot notation

Traditional Usage

The traditional approach uses the model: resolver to describe and instantiate a fitting model in a single, declarative string:

  • fitting_model: "model:PolynomialFittingModel:degree=2" → a 2nd-degree polynomial

  • context_keyword: "fit_coefficients" → where results are stored in the context (the context must provide x_values and y_values arrays)

At runtime, ModelFittingContextProcessor reads the arrays from payload.context, fits the model, and stores the coefficients under payload.context["fit_coefficients"].

Flexible Parameter Mapping

When independent_var_key and/or dependent_var_key parameters are specified, the system automatically creates a specialized processor that can extract data using custom parameter names and nested paths.

Configuration Examples

Basic flexible parameter mapping:

pipeline:
  nodes:
    - processor: ModelFittingContextProcessor
      parameters:
        fitting_model: "model:PolynomialFittingModel:degree=1"
        independent_var_key: "time_values"
        dependent_var_key: "measurements"
        context_keyword: "time_series_fit"

Nested path extraction:

pipeline:
  nodes:
    - processor: ModelFittingContextProcessor
      parameters:
        fitting_model: "model:PolynomialFittingModel:degree=1"
        independent_var_key: "t_values"
        dependent_var_key: "gaussian_fit_parameters.std_dev_x"
        context_keyword: "std_dev_coefficients"

Slicer integration:

pipeline:
  nodes:
    - processor: ModelFittingContextProcessor
      parameters:
        fitting_model: "model:PolynomialFittingModel:degree=1"
        independent_var_key: "slice_indices"
        dependent_var_key: "aggregated_data.mean_values"
        context_keyword: "trend_analysis"

Multiple fitting operations:

pipeline:
  nodes:
    # Fit standard deviation trend
    - processor: ModelFittingContextProcessor
      parameters:
        fitting_model: "model:PolynomialFittingModel:degree=1"
        independent_var_key: "t_values"
        dependent_var_key: "gaussian_fit_parameters.std_dev_x"
        context_keyword: "std_dev_trend"

    # Fit orientation trend  
    - processor: ModelFittingContextProcessor
      parameters:
        fitting_model: "model:PolynomialFittingModel:degree=1"
        independent_var_key: "t_values"
        dependent_var_key: "gaussian_fit_parameters.angle"
        context_keyword: "orientation_trend"

Data Structure Examples

The flexible parameter mapping supports various data structure formats:

Single Dictionary Format:

data = {
    "t_values": [1.0, 2.0, 3.0, 4.0, 5.0],
    "gaussian_fit_parameters": {
        "std_dev_x": [0.1, 0.2, 0.3, 0.4, 0.5],
        "std_dev_y": [0.15, 0.25, 0.35, 0.45, 0.55],
        "angle": [10, 15, 20, 25, 30]
    }
}

List of Dictionaries Format (Slicer Output):

data = {
    "t_values": [1.0, 2.0, 3.0, 4.0, 5.0],
    "gaussian_fit_parameters": [
        {"std_dev_x": 0.1, "std_dev_y": 0.15, "angle": 10},
        {"std_dev_x": 0.2, "std_dev_y": 0.25, "angle": 15},
        {"std_dev_x": 0.3, "std_dev_y": 0.35, "angle": 20},
        {"std_dev_x": 0.4, "std_dev_y": 0.45, "angle": 25},
        {"std_dev_x": 0.5, "std_dev_y": 0.55, "angle": 30}
    ]
}

Integration with Pipeline Components

Slicer Integration

The flexible parameter mapping is particularly useful when working with slicer outputs. The system automatically detects and handles data from slice aggregators:

- processor: SliceAggregatorContextProcessor
  parameters:
    # ... slicer configuration

- processor: ModelFittingContextProcessor
  parameters:
    fitting_model: "model:PolynomialFittingModel:degree=1"
    independent_var_key: "slice_indices"
    dependent_var_key: "aggregated_data.mean_values"
    context_keyword: "trend_analysis"

Multiple Fitting Operations

You can perform multiple fits on different aspects of the same data by configuring multiple processors with different dependent variable paths:

# Fit standard deviation trend
- processor: ModelFittingContextProcessor
  parameters:
    fitting_model: "model:PolynomialFittingModel:degree=1"
    independent_var_key: "t_values"
    dependent_var_key: "gaussian_fit_parameters.std_dev_x"
    context_keyword: "std_dev_trend"

# Fit orientation trend
- processor: ModelFittingContextProcessor
  parameters:
    fitting_model: "model:PolynomialFittingModel:degree=1"
    independent_var_key: "t_values"
    dependent_var_key: "gaussian_fit_parameters.angle"
    context_keyword: "orientation_trend"

Technical Implementation

Automatic Factory Selection

When the pipeline node factory detects independent_var_key and/or dependent_var_key parameters in a ModelFittingContextProcessor configuration, it automatically:

  1. Creates a specialized processor class using _model_fitting_processor_factory()

  2. Configures the processor to expect the specified parameter names

  3. Sets up nested path extraction for the dependent variable

  4. Maintains all standard functionality (output keys, error handling, etc.)

Nested Path Resolution

The system uses dot notation to traverse nested structures:

  • "data.field" → data["field"] or data.field

  • "measurements.values" → measurements["values"] or measurements.values

  • Works with both dictionary access and attribute access

Error Handling

The system provides comprehensive error handling for:

  • Missing parameters (clear error messages for required parameters)

  • Invalid paths (detailed path resolution errors with context)

  • Data type mismatches (validation of expected data structures)

  • Empty data (handling of edge cases with empty or null data)

Run it (inspect, then execute)

1) Inspect the pipeline (pre-flight checks)

Traditional usage:

semantiva inspect tests/pipeline_model_fitting.yaml --extended

Flexible parameter mapping:

semantiva inspect tests/pipeline_model_fitting_flexible.yaml --extended
semantiva inspect tests/pipeline_model_fitting_nested_path.yaml --extended

You should see the canonical identities (PipelineId, per-node node_uuid) and the configured parameters for the fitting step. See Introspection & Validation.

2) Execute from the CLI with initial context

Traditional usage (provide arrays via --context):

semantiva run tests/pipeline_model_fitting.yaml \
  --context x_values=[-1.0,0.0,1.0,2.0] \
  --context y_values="[1.0,1.0,2.5,5.0]"

Flexible parameter mapping:

semantiva run tests/pipeline_model_fitting_flexible.yaml \
  --context time_values=[0,1,2,3,4] \
  --context measurements="[1.1,1.9,3.1,3.9,5.1]"

Nested path example:

semantiva run tests/pipeline_model_fitting_nested_path.yaml \
  --context t_values=[1,2,3,4,5] \
  --context 'gaussian_fit_parameters={"std_dev_x":[0.1,0.2,0.3,0.4,0.5]}'

The resulting coefficients are written to the specified context keyword.

3) Visualize the config

semantiva-studio-viewer serve-pipeline tests/pipeline_model_fitting.yaml --port 8000
# open http://127.0.0.1:8000 and click the fitting node to see its parameters and node_uuid

Link-outs: - Pipelines in Semantiva (Payload and dual channels) - Extending Semantiva (Registries & Extensions) (model: resolver and extensions) - Semantiva Studio Viewer (serve and export diagrams) - Context Processors (Context processor fundamentals) - Data Processors (Data slicer integration)

Example YAML Configurations

Traditional Usage

pipeline:
  nodes:
    - processor: ModelFittingContextProcessor
      parameters:
        fitting_model: "model:PolynomialFittingModel:degree=2"
        context_keyword: "fit_coefficients"

Flexible Parameter Mapping

pipeline:
  nodes:
    - processor: ModelFittingContextProcessor
      parameters:
        fitting_model: "model:PolynomialFittingModel:degree=1"
        independent_var_key: "time_values"
        dependent_var_key: "measurements"
        context_keyword: "time_series_fit"

Nested Path Extraction

pipeline:
  nodes:
    - processor: ModelFittingContextProcessor
      parameters:
        fitting_model: "model:PolynomialFittingModel:degree=1"
        independent_var_key: "t_values"
        dependent_var_key: "gaussian_fit_parameters.std_dev_x"
        context_keyword: "std_dev_coefficients"

Slicer Integration

pipeline:
  nodes:
    - processor: ModelFittingContextProcessor
      parameters:
        fitting_model: "model:PolynomialFittingModel:degree=1"
        independent_var_key: "slice_indices"
        dependent_var_key: "aggregated_data.mean_values"
        context_keyword: "trend_analysis"

Multiple Operations

pipeline:
  nodes:
    # Fit standard deviation trend
    - processor: ModelFittingContextProcessor
      parameters:
        fitting_model: "model:PolynomialFittingModel:degree=1"
        independent_var_key: "t_values"
        dependent_var_key: "gaussian_fit_parameters.std_dev_x"
        context_keyword: "std_dev_trend"

    # Fit orientation trend  
    - processor: ModelFittingContextProcessor
      parameters:
        fitting_model: "model:PolynomialFittingModel:degree=1"
        independent_var_key: "t_values"
        dependent_var_key: "gaussian_fit_parameters.angle"
        context_keyword: "orientation_trend"

Programmatic Usage

>>> from semantiva.configurations import load_pipeline_from_yaml
>>> from semantiva.pipeline import Pipeline
>>>
>>> # Traditional usage
>>> nodes = load_pipeline_from_yaml("tests/pipeline_model_fitting.yaml")
>>> p = Pipeline(nodes)
>>> # Requires x_values/y_values in context
>>>
>>> # Flexible parameter mapping
>>> nodes = load_pipeline_from_yaml("tests/pipeline_model_fitting_flexible.yaml")
>>> p = Pipeline(nodes)
>>> # Requires time_values/measurements in context
>>>
>>> # Using the factory directly
>>> from semantiva.workflows.fitting_model import _model_fitting_processor_factory
>>> ProcessorClass = _model_fitting_processor_factory(
...     independent_var_key="time",
...     dependent_var_key="data.measurements",
...     context_keyword="analysis_results"
... )
>>> processor = ProcessorClass()

API Reference

semantiva.workflows.fitting_model._model_fitting_processor_factory(independent_var_key, dependent_var_key, context_keyword)[source]

Autodoc

class semantiva.workflows.fitting_model.FittingModel[source]

Bases: ABC, Generic[X, Y]

abstract fit(x_values, y_values)[source]
class semantiva.workflows.fitting_model.ModelFittingContextProcessor(logger=None)[source]

Bases: ContextProcessor

CONTEXT_OUTPUT_KEY = 'fit.parameters'
classmethod context_keys()[source]
classmethod with_context_keyword(key)[source]
class semantiva.workflows.fitting_model.PolynomialFittingModel(degree)[source]

Bases: FittingModel[float, float]

fit(x_values, y_values)[source]