Evaluate on CdSprites+ dataset
- multimodal_compare.eval.eval_cdsprites.calculate_cross_coherency(model_exp, classifiers)
Calculates the cross-coherency accuracy for the given model (Img -> Txt and Txt -> Img)
- Parameters:
model (object) – multimodal VAE
- Returns:
mean cross accuracies
- Return type:
dict
- multimodal_compare.eval.eval_cdsprites.calculate_joint_coherency(model_exp, classifiers)
Calculates the joint-coherency accuracy for the given model
- Parameters:
model (object) – multimodal VAE
- Returns:
mean joint accuracy
- Return type:
dict
- multimodal_compare.eval.eval_cdsprites.check_cross_sample_correct(testtext, m_exp, classifiers=None, reconimage=None, recontext=None)
Detects the features in images/text and checks if they are coherent
- Parameters:
testtext (str) – ground truth text input
reconimage (ndarray) – reconstructed image
recontext (str) – reconstructed text
- Returns:
returns whether the sample is completely correct, how many features are ok, how many letters are ok
- Return type:
tuple(Bool, float32, float32)
- multimodal_compare.eval.eval_cdsprites.count_same_letters(a, b)
Counts how many characters are the same in two strings.
- Parameters:
a (str) – string 1
b (str) – string 2
- Returns:
number of matching characters
- Return type:
int
- multimodal_compare.eval.eval_cdsprites.eval_all(model_exp, classifiers)
- multimodal_compare.eval.eval_cdsprites.eval_cdsprites_over_seeds(parent_dir)
- multimodal_compare.eval.eval_cdsprites.eval_single_model(m_exp)
- multimodal_compare.eval.eval_cdsprites.eval_with_classifier(classifier, image, att)
- multimodal_compare.eval.eval_cdsprites.fill_cats(text_image, image_text, joint, data)
- multimodal_compare.eval.eval_cdsprites.find_in_list(target, source)
- Parameters:
target –
source –
- Returns:
- Return type:
- multimodal_compare.eval.eval_cdsprites.get_all_classifiers(level)
- multimodal_compare.eval.eval_cdsprites.get_attribute(attribute, txt)
- Parameters:
attribute –
txt –
- Returns:
- Return type:
- multimodal_compare.eval.eval_cdsprites.get_attribute_from_recon(attribute, txt, m_exp)
- Parameters:
attribute –
txt –
- Returns:
- Return type:
- multimodal_compare.eval.eval_cdsprites.get_mean_stats(list_of_stats, percentage=True)
Returns a list of means for a nested list with accuracies
- Parameters:
list_of_stats (list) – multiple lists with accuracies
percentage (bool) – whether to report the number as percent (True) or fraction (False)
- Returns:
a list of means of the accuracies
- Return type:
list
- multimodal_compare.eval.eval_cdsprites.get_mod_mappings(mod_dict)
- multimodal_compare.eval.eval_cdsprites.image_to_text(imgs, model_exp)
Reconstructs image from the text input using the provided model :param imgs: list of images to reconstruct :type imgs: list :param model: model object :type model: object :param path: where to save the outputs :type path: str :return: returns reconstructed images and texts :rtype: tuple(list, list)
- multimodal_compare.eval.eval_cdsprites.load_classifier(level: int, class_type: str)
- multimodal_compare.eval.eval_cdsprites.load_images(path)
Loads .png images from a dir path
- Parameters:
path (str) – path to the folder
- Returns:
list of ndarrays
- Return type:
list
- multimodal_compare.eval.eval_cdsprites.manhattan_distance(a, b)
Calculates the Manharran distance between two vectors
- Parameters:
a (tuple) – vec 1
b (tuple) – vec 2
- Returns:
distance
- Return type:
float
- multimodal_compare.eval.eval_cdsprites.search_att(txt, source, idx=None, indices=None)
- Parameters:
txt –
source –
idx –
indices –
- Returns:
- Return type:
- multimodal_compare.eval.eval_cdsprites.text_to_image(text, model_exp)
Reconstructs text from the image input using the provided model :param text: list of strings to reconstruct :type text: list :param model: model object :type model: object :param path: where to save the outputs :type path: str :return: returns reconstructed images and also texts :rtype: tuple(list, list)
- multimodal_compare.eval.eval_cdsprites.try_retrieve_atts(txt, m_exp)
- Parameters:
txt –
- Returns:
- Return type: