Cyrus Import Mail Invalid Header - Revision history

Ivc at 19:43, 30 May 2016

2016-05-30T19:43:41Z

← Older revision		Revision as of 19:43, 30 May 2016
Line 7:		Line 7:
	== Finding problematic messages ==		== Finding problematic messages ==

	The easiest way to find the problematic messages is to purge the ones that imports fine and hash out the remaining messages. I did this in Thunderbird by using the Remove Duplicate Messages Alternate add-ons. Also the ImportExportTools add-ons comes in handy.		Using Thunderbird: The easiest way to find the problematic messages is to purge the ones that imports fine and hash out the remaining messages. I did this in Thunderbird by using the Remove Duplicate Messages Alternate add-ons. Also the ImportExportTools add-ons comes in handy.

	# Prepare the folder you want to move over to the Cyrus IMAP server		# Prepare the folder you want to move over to the Cyrus IMAP server

Ivc at 19:27, 4 February 2016

2016-02-04T19:27:14Z

← Older revision		Revision as of 19:27, 4 February 2016
Line 1:		Line 1:
	This is a common issue people are having when importing mail messages from other services or desktop clients. The message normally shown in the console when importing the messages is:		This is a common issue people are having when importing mail messages from other services or desktop clients to Cyrus IMAP 2.4.16. The message normally shown in the console when importing the messages is:

	Message contains invalid header		Message contains invalid header

Ivc at 19:11, 4 February 2016

2016-02-04T19:11:48Z

← Older revision		Revision as of 19:11, 4 February 2016
Line 20:		Line 20:
	# View source of the message and inspect the header and fine what is different from a working message		# View source of the message and inspect the header and fine what is different from a working message

	== Headers containing '>>From ~~header~~' ==		== Headers containing '>>From ' ==

	In my case, it turned out that the messages with headers starting with:		In my case, it turned out that the messages with headers starting with:

Ivc at 19:11, 4 February 2016

2016-02-04T19:11:28Z

← Older revision		Revision as of 19:11, 4 February 2016
Line 40:		Line 40:
	# Right-click on the folder you want to fix and select Export as mbox file		# Right-click on the folder you want to fix and select Export as mbox file
	# Open a terminal window (Mac or Linux) and run the following command to see which lines return		# Open a terminal window (Mac or Linux) and run the following command to see which lines return
	## cat INBOX.mbox \| grep ">>From "		#* cat INBOX.mbox \| grep ">>From "
	## Result should be lots of line similar to: >>From vbei@cox.net Fri Feb 20 16:10:42 2004		#* Result should be lots of line similar to: >>From vbei@cox.net Fri Feb 20 16:10:42 2004
	# Now, to change all the instances of ">>From " to ">From" (which is fine by Cyrus)		# Now, to change all the instances of ">>From " to ">From" (which is fine by Cyrus)
	## cat INBOX.mbox \| sed -e 's/^>>From \(.\)\([0-9]\) 20\(.\)/>From \1\2 20\3/g' > INBOX-fixed.mbox		## cat INBOX.mbox \| sed -e 's/^>>From \(.\)\([0-9]\) 20\(.\)/>From \1\2 20\3/g' > INBOX-fixed.mbox
	# The new file should be ready to import, some useful commands to check and verify		# The new file should be ready to import, some useful commands to check and verify
	## cat INBOX.mbox \| grep ">>From "\|wc -l		#* cat INBOX.mbox \| grep ">>From "\|wc -l
	## tail -100 INBOX-fixed.mbox		#* tail -100 INBOX-fixed.mbox
	# Import the fixed mbox in Thunderbird by right-clicking the Local folders and Import mbox file		# Import the fixed mbox in Thunderbird by right-clicking the Local folders and Import mbox file
	# Finally, drag the newly imported folder over to the Cyrus IMAP folder where you want the messages to be stored		# Finally, drag the newly imported folder over to the Cyrus IMAP folder where you want the messages to be stored

Ivc at 19:11, 4 February 2016

2016-02-04T19:11:13Z

← Older revision		Revision as of 19:11, 4 February 2016
Line 26:		Line 26:
	">>From "		">>From "

	~~and~~		or

	">>>From "		">>>From "

Ivc at 19:09, 4 February 2016

2016-02-04T19:09:57Z

← Older revision		Revision as of 19:09, 4 February 2016
Line 20:		Line 20:
	# View source of the message and inspect the header and fine what is different from a working message		# View source of the message and inspect the header and fine what is different from a working message

	== >>From header ==		== Headers containing '>>From header' ==

	In my case, it turned out that the messages with headers starting with:		In my case, it turned out that the messages with headers starting with:

Ivc: Created page with "This is a common issue people are having when importing mail messages from other services or desktop clients. The message normally shown in the console when importing the mess..."

2016-02-04T19:09:29Z

Created page with "This is a common issue people are having when importing mail messages from other services or desktop clients. The message normally shown in the console when importing the mess..."

New page

This is a common issue people are having when importing mail messages from other services or desktop clients. The message normally shown in the console when importing the messages is:

Message contains invalid header

This is because Cyrus IMAPD is strict and requires the message header to be within a certain bound.

== Finding problematic messages ==

The easiest way to find the problematic messages is to purge the ones that imports fine and hash out the remaining messages. I did this in Thunderbird by using the Remove Duplicate Messages Alternate add-ons. Also the ImportExportTools add-ons comes in handy.

# Prepare the folder you want to move over to the Cyrus IMAP server
# Let it copy and wait for the transfer to end with the Invalid Header error
# Make a copy of the original messages folder
# Copy back all the messages on the Cyrus server back to the new copied folder
# In the Duplicate Messages Alternate add-on options, change the filtering to "To", "From", "Message-Id" and "Subject"
# Right-click on the newly copied folder with all the Cyrus messages also included
# A list of lots duplicates will be displayed, click No Copies, and click Ok to purge the ones that did copy to the server
# You're left with a folder with mostly messages that did not copy to the Cyrus server
# Try to copy a few of the messages until you get the Invalid Header error
# View source of the message and inspect the header and fine what is different from a working message

== >>From header ==

In my case, it turned out that the messages with headers starting with:

">>From "

and

">>>From "

Was detected to be invalid and thus report an "Invalid Header" error.

== Fixing mbox ==

By simply exporting all the messages to a mbox-file, parsing all the lines in the mbox and imorting the mbox again fixed the problem.

Install the ImportExportTools add-on.

# Right-click on the folder you want to fix and select Export as mbox file
# Open a terminal window (Mac or Linux) and run the following command to see which lines return
## cat INBOX.mbox | grep ">>From "
## Result should be lots of line similar to: >>From vbei@cox.net Fri Feb 20 16:10:42 2004
# Now, to change all the instances of ">>From " to ">From" (which is fine by Cyrus)
## cat INBOX.mbox | sed -e 's/^>>From \(.*\)\([0-9]\) 20\(.*\)/>From \1\2 20\3/g' > INBOX-fixed.mbox
# The new file should be ready to import, some useful commands to check and verify
## cat INBOX.mbox | grep ">>From "|wc -l
## tail -100 INBOX-fixed.mbox
# Import the fixed mbox in Thunderbird by right-clicking the Local folders and Import mbox file
# Finally, drag the newly imported folder over to the Cyrus IMAP folder where you want the messages to be stored