i have been trying to use sort | uniq when combining combo lists to sort n remove duplicates but i keep getting following error

sort: mbrtowc error: Illegal byte sequence

i searched online n tried LCTYPE=C but get same error n am not sure what to do anyone have ideas

maybe print out where error occurs n remove line somehow
idk any ideas thx

maybe something to do with the encoding

(September 14, 2022, 08:26 PM)Eprincess Wrote: maybe something to do with the encoding

That's my though to although I don't think I've ever actually had this error but then again I haven't ran sort with LANG=C before. My guess is there's some Unicode text in that file that the program can't parse since you enforcing ascii encoding.

would also assume some encoding issue

give an example or some code snippet to look

thanls