folktexts.cli package

Submodules

folktexts.cli.download_models module

folktexts.cli.download_models.is_bf16_compatible()[source]

Checks if the current environment is bfloat16 compatible.

Return type:: bool

folktexts.cli.download_models.main()[source]

folktexts.cli.download_models.setup_arg_parser()[source]

Return type:: ArgumentParser

folktexts.cli.eval_feature_importance module

folktexts.cli.eval_feature_importance.compute_feature_importance(llm_clf, dataset, scorer, results_dir, fit_threshold=None, seed=42)[source]

Return type:: dict

folktexts.cli.eval_feature_importance.main()[source]

folktexts.cli.eval_feature_importance.parse_feature_importance(results, columns)[source]

Parse the results dictionary of sklearn’s permutation_importance.

Return type:: dict

folktexts.cli.eval_feature_importance.setup_arg_parser()[source]

Return type:: ArgumentParser

folktexts.cli.experiments module

General constants and helper classes to run the main experiments on htcondor.

class folktexts.cli.experiments.Experiment(executable_path, env_vars='', kwargs=<factory>, job_cpus=4, job_gpus=0, job_memory_gb=62, job_gpu_memory_gb=30, job_bid=25)[source]

Bases: object

A generic experiment to run on the cluster.

env_vars: str = ''

executable_path: str

classmethod get_all_experiments()[source]

hash()[source]

Generate a hexadecimal hash that uniquely identifies the experiment’s arguments.

Return type:: str

job_bid: int = 25

job_cpus: int = 4

job_gpu_memory_gb: int = 30

job_gpus: int = 0

job_memory_gb: int = 62

kwargs: dict

to_dict()[source]

Return type:: dict

folktexts.cli.experiments.launch_experiment_job(exp)[source]

folktexts.cli.launch_experiments_htcondor module

Launch htcondor jobs for all ACS benchmark experiments.

folktexts.cli.launch_experiments_htcondor.main()[source]

folktexts.cli.launch_experiments_htcondor.make_llm_clf_experiment(executable_path, model_name, task, results_dir, env_vars_str='', **kwargs)[source]

Create an experiment object to run.

Return type:: Experiment

folktexts.cli.launch_experiments_htcondor.setup_arg_parser()[source]

Return type:: ArgumentParser

folktexts.cli.rerun_experiment module

Helper script to re-run a single experiment locally.

folktexts.cli.rerun_experiment.setup_arg_parser()[source]

Return type:: ArgumentParser

folktexts.cli.run_acs_benchmark module

Runs the LLM calibration benchmark from the command line.

folktexts.cli.run_acs_benchmark.main()[source]: Prepare and launch the LLM-as-classifier experiment using ACS data.

folktexts.cli.run_acs_benchmark.setup_arg_parser()[source]

Return type:: ArgumentParser