“Encountered unexpected character 'ï'” error serializing JSON

When trying to deserialize JSON-encoded data from a file using the DataContractJsonSerializer, you may run into the following error:

System.Runtime.Serialization.SerializationException: There was an error deserializing the object of type
ConsoleApplication1.Person. Encountered unexpected character 'ï'. --->
System.Xml.XmlException: Encountered unexpected character 'ï'.
   at System.Xml.XmlExceptionHelper.ThrowXmlException(XmlDictionaryReader reader
, XmlException exception)
   at System.Runtime.Serialization.Json.XmlJsonReader.ReadAttributes()
   at System.Runtime.Serialization.Json.XmlJsonReader.ReadNonExistentElementName (StringHandleConstStringType elementName)
   at System.Runtime.Serialization.Json.XmlJsonReader.Read()
   at System.Xml.XmlBaseReader.IsStartElement()
   at System.Xml.XmlBaseReader.IsStartElement(XmlDictionaryString localName, XmlDictionaryString namespaceUri)
   at System.Runtime.Serialization.XmlReaderDelegator.IsStartElement(XmlDictionaryString localname, XmlDictionaryString ns)
   at System.Runtime.Serialization.XmlObjectSerializer.IsRootElement(XmlReaderDelegator reader, DataContract contract, XmlDictionaryString name, XmlDictionaryString ns)
   at System.Runtime.Serialization.Json.DataContractJsonSerializer.InternalIsStartObject(XmlReaderDelegator reader)
   at System.Runtime.Serialization.Json.DataContractJsonSerializer.InternalReadObject(XmlReaderDelegator xmlReader, Boolean verifyObjectName)
   at System.Runtime.Serialization.XmlObjectSerializer.InternalReadObject(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContract Resolver)
   at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
   --- End of inner exception stack trace ---
   at System.Runtime.Serialization.XmlObjectSerializer.ReadObjectHandleExceptions(XmlReaderDelegator reader, Boolean verifyObjectName, DataContractResolver dataContractResolver)
    at System.Runtime.Serialization.Json.DataContractJsonSerializer.ReadObject(XmlDictionaryReader reader)
   at System.Runtime.Serialization.Json.DataContractJsonSerializer.ReadObject(Stream stream)
   at ConsoleApplication1.Program.Main(String[] args) in e:\AzureSamples\JSONSerializer\ConsoleApplication1\ConsoleApplication1\Program.cs

One of the reasons for this could be that the input file that contains the JSON-encoded data is created with a binary encoding or it has a Byte Order Mark(BOM) byte sequence for a binary encoded file.
For e.g. The UTF-8 representation of the BOM is the byte sequence (0xEF,0xBB,0xBF) in the beginning of the file.
Note: You will see this if you created a .JSON file(or a binary file) using visual studio.

To verify this, you can open the file using a binary editor like visual studio. To open the Binary Editor, first choose File | New | File from the main menu, select the file you want to edit, then click on the drop arrow next to the Open button, and choose Open With | Binary Editor.

 

One way to solve the issue is to delete the BOM byte sequence (0xEF,0xBB,0xBF) in the beginning of the file.

https://connect.microsoft.com/VisualStudio/feedback/details/356750/datacontractjsonserializer-fails-with-non-ansi-characters has the following which might be relevant:

Please note that DataContractJsonSerializer only supports the following encodings: UTF-8, UTF-16LE and UTF-16BE. For this to work, you need to do something like:     
     
MemoryStream ms = new MemoryStream(Encoding.UTF8.GetBytes(panoramio));
DataContractJsonSerializer dcjs = new DataContractJsonSerializer(typeof(PhotoData));
PhotoData pd = (PhotoData)dcjs.ReadObject(ms);

Comments

• Anonymous
  July 09, 2014
  Thank you very much! I've been looking for days how to solve this issue and didn't found the answer, until now. Does this happen with ALL of the JSON files created through Visual Studio? It's a bit frustrating that the mistake was with the way the serializer encoded the data, not a mistake by myself (I'm a beginner in programming so that's the common thing). Is there a way to avoid this?
• Anonymous
  August 20, 2014
  Thanks!  This helped me in a little test harness I'm using.  Very good tip.
• Anonymous
  November 04, 2014
  The comment has been removed
• Anonymous
  February 05, 2015
  I had the same problem. I solved it like this. I used free NotePad++ . I used the Encoding tool there and encoded the file using "Encode with UTF-8 without BOM". This one click approach resolved the issue.
• Anonymous
  May 27, 2015
  Muchisimas gracias!!! fue de mucha ayuda!! :D
• Anonymous
  May 24, 2016
  The comment has been removed
  • Anonymous
    November 11, 2016
    Thanks for you comments Don. Glad that it helped you.
• Anonymous
  August 27, 2016
  The comment has been removed