nxlog: how to convert from windows-1252 to utf-8 ?

[_asp_]

Hi,

I have a logfile which is in charset windows-1252.

After combining lines in nxlog using multilne module i get following errors in logstash:

Code: Select all

message=>"Received an event that has a different character encoding than you configured."

If I do an offline conversion with iconv from windows-1252 to utf-8 everything works fine.

But how can I do that in nxlog in realtime?

I tried the following, but the error stays:

Code: Select all

<Extension charconv>
    Module      xm_charconv
    AutodetectCharsets utf-8, euc-jp, utf-16, utf-32, iso8859-2, ucs-2le, ascii, utf-7, windows-1252
</Extension>

<Extension multilineVicHandler>
	Module xm_multiline
	HeaderLine /INFO: VIC_HANDLER Service: Start/
</Extension>


<Input vichandler>
	Module im_file
	File "C:/lw_g/amest/logfile/vichandler/vichandler.log"
	
	SavePos true
	#Exec if ($raw_event !~ /.*originator.*Proc-Time.*/) drop();
	
	#enabling multilining
	InputType multilineVicHandler

	SavePos TRUE
	Exec $Message = $raw_event;
	Exec $LogFile = 'vichandler';


<Output out>
	Module om_tcp
	#Host 139.1.117.45
	Host server
	Port 2057
		
	Exec $tmpmessage = $Message; delete($Message); rename_field("tmpmessage","message");
	Exec $raw_event = to_json();
	
	#Exec	to_json();
	
	
	# Uncomment for debug output
	Exec file_write('c:\nxlog\nxlog_debug_output.log', $raw_event + "\n");
</Output>
</Input>

[mcapra]

Have you tried defining the charset within your input's parameters? Like so:

Code: Select all

codec => plain {
       charset => "Windows-1252"
     }

Need to be a bit careful with the plain codec though since it can overwrite the existing framing of a message.