Similarity Featurizer optimized for Prefix Filtering
If set to true, the algorithm will automatically calculate token weights. Default token weights are defined based on token idf values.
Adding weights into the join might lead to more reliable pair comparisons but could add overhead to the algorithm. However, smart optimizations such as Prefix Filtering used in some implementations of AnnotatedSimilarityFeaturizer might actually reduce overhead if there is an abundance of common tokens in the dataset.
Perform a Broadcast Join