Validation

actantial.validation.compare_annotations(dfs, names=None, id_column='id', metric='krippendorff_alpha', actant_columns=ACTANTS, verbose=False)

Compare multiple sets of actantial annotations and compute an agreement metric.

Aligns all DataFrames on id_column (outer join), then computes the chosen metric for each actant. Pairwise metrics (accuracy, F1) iterate over all annotator pairs and average the results. Krippendorff's alpha uses all coders simultaneously and handles missing values natively.

Parameters:

Name	Type	Description	Default
dfs	List[DataFrame]	List of DataFrames with actantial annotations (one per annotator).	required
names	Optional[List[str]]	Annotator names, one per DataFrame. Defaults to coder_0, coder_1, etc.	None
id_column	str	Column name used to align rows across DataFrames.	'id'
metric	str	Agreement metric. One of "accuracy", "f1_micro", "f1_macro", "f1_weighted", "krippendorff_alpha".	'krippendorff_alpha'
actant_columns	list	Actant columns to compare. Defaults to the full set defined in the package config.	ACTANTS
verbose	bool	If True, print warnings about dropped rows.	False

Returns:

Type	Description
DataFrame	DataFrame with actants as rows (plus an "avg" row). Columns are annotator pairs plus "avg" for pairwise metrics, or a single "alpha" column for Krippendorff.