2 new options - ignore all dupes and remove blanks