WebMatrix UTF-8 support missing?
Last post Feb 03, 2011 05:23 PM by alisonl
Feb 02, 2011 03:31 PM|luminarious|LINK
me, as it messes up my existing UTF-8 files and makes it harder to share my work with other developers who also expect UTF-8 files.
I hope remeding this is not too far away on the roadmap.
Also, this may be an edge case, but the UI rendering is a bit on the ugly side when Large Text (125%) mode is used in Windows 7.
Feb 02, 2011 05:56 PM|simtan|LINK
Thank you for trying WebMatrix! You are correct that the default behavior of WebMatrix is to save files with your local default encoding (which may be ANSI). However, in all file types
except for PHP, inserting special characters outside of the local encoding set (i.e. Unicode characters) should trigger a prompt to save in UTF-8.
If you're experiencing this problem in PHP files, however, it is because we currently cannot save in UTF-8 encoding without some sort of "hint". One such hint you can provide is to insert a <meta charset="utf-8"/> tag in the file, to signify
that the page is actually UTF-8. The reason for this is because we cannot use our normal method for recording this information (a byte-order mark) in PHP, as most PHP engines are incompatible with it.
We are currently looking into another workaround for the PHP case - please let us know if you are experiencing this with other file types as well.
Thanks for your feedback!
Feb 03, 2011 03:21 AM|luminarious|LINK
Thank you for the quick reply!
Is there any specific reason UTF-8 isn't the default choice? Because I've been using Notepad2 (http://www.flos-freeware.ch/notepad2.html - source available) as my main editor for a long time and have had absolutely no issues anywhere with UTF-8 as the default
encoding. It works with PHP as well as anything else.
When I created a CSS file and added an Õ character, it was saved as UTF-8 BOM. But a JS file was saved as ANSI with the same added character. Why is that?
Feb 03, 2011 03:35 AM|HCamper|LINK
Hope you get his fixed.
Feb 03, 2011 11:10 AM|rlucero|LINK
The suggestion that you posted above has nothing to do with the problem that Luminarios is facing. His issue is that for some reason, WebMatrix is changing the code page information from UTF-8 to ANSI when the editor saves the file.
It has nothing to do with how IIS or IIS Express is serving the pages.
Feb 03, 2011 03:02 PM|alisonl|LINK
Hello, I had a quick question. When you say you created a new CSS and a new JS file, can you clarify what you used to create them? Did you do it within WebMatrix via the New File dialog? Our JS file template comes as UTF-8 BOM, so if you did using that
method, I agree that we should expect it to also save as UTF-8 BOM.
However, if the file was originally created in ANSI, we try to preserve that encoding on save unless we detect characters that prevent us from doing so. For example, if you introduced a Unicode character. In this case, we should show a prompt on save asking
if you would like to save in UTF-8.
Feb 03, 2011 03:07 PM|Yishai G.|LINK
Thanks for using WebMatrix and noticing this very important aspect.
I wanted to provide a little correction to the explanation above about how encoding works. For the sake of simplicity I'm ignoring the PHP files handling.
In short WebMatrix does not switch encoding from ANSI or Default Code Page, and it tries to preserve the initial encoding the file was started with. So in the example you presented where you took a JS file added a non English char and saved it, one of three
things can happen
1. The file was encoded in UTF-8 when you started (as in it had a BOM in it, as all files from out templates have), and hence it will be resaved in UTF-8.
2. The file was not encoded in UTF-8. In this case if the char you added is part of your default code page, it will maintain the original encoding. If it's from a different code page, we will save it in UTF-8.
So as long as you start coding from our templates, or start with files that are UTF-8 encoded (as in they have a BOM), the files will be saved in UTF-8.
So to summarize:
If you start with UTF-8 you will always get UTF-8 on saving.
If you start with a local code page, and all you chars are within that local code page, we will not switch to UTF-8.
If the file cannot be saved in the local code page, we will save it in UTF-8.
I hope this clarifies the intent of how the editor is supposed to save files. If you have any further questions or examples where the above described process is not working properly, please let us know.
The WebMatrix team
Feb 03, 2011 03:10 PM|Yishai G.|LINK
Feb 03, 2011 03:31 PM|HCamper|LINK
Hello @ Yishai,
If I am reading this correctly that the
advise is to pick an editor that properly follows the rules
that you outlined.
So if that it true NotePad on Windows 7 will work but NotePad does not offer syntax
and tag matching as features.
So what have you found that works as a NotePad replacement that has syntax tag matching?
I think we can work around the idea that this maybe this would be third part product
and will have no "Microsoft" support as part of suggested recommendations.
The answer could be a combination of following the Rules and a personal preference.
Thank You for Your Time And Attention,
Feb 03, 2011 04:13 PM|Yishai G.|LINK
What I was actually saying is that WebMatrix Editor does support and follow all these rules. And I would suggest using it :)
If you find any issues with it, this is an excellent place to report them. We are here to listen, explain and eventually fix the next releases as appropriate.
Feb 03, 2011 04:24 PM|luminarious|LINK
Alison, yes, I created them from WebMatrix New File dialog. So it all makes sense if the template has UTF-8 BOM.
Yishai, yes, I installed the WPI pack and use the WebMatrix that was included. WebMatrix doesn't seem to have an About dialog, so I don't know the version number.
I think I understand now. In my current editor I'm using the 'UTF-8 without BOM' setting as default because I often do PHP work. WebMatrix doesn't find the BOM and sees the files as ANSI. My preferred solution would of course be for WebMatrix to adopt the
UTF-8 sans BOM as an optional default setting for all files. This kind of files are not at all uncommon, in my experience.
Feb 03, 2011 04:44 PM|Yishai G.|LINK
Ok then, That makes things very clear. And I wanted to thank you for your insight about cases were users will use UTF-8 with BOM.
Note that UTF-8 without BOM is not a recommended encoding for .aspx files (and .cshtml files etc.) and hence making a sweeping decision like that was not possible.
We did specifically make a change to encode PHP files as UTF-8 without BOM, so that should alleviate PHP issue and still keep PHP with BOM for all other files.
This feedback is very valueable towards our next releases, and I wanted to thank you again for taking the time and using WebMatrix.
Feb 03, 2011 04:57 PM|luminarious|LINK
Feb 03, 2011 04:57 PM|alisonl|LINK
Thanks luminarious for the extra info! And yes, your understanding of what is happening sounds like the right explaination of why WebMatrix did not save your files as UTF-8 BOM.
To find the version number, you can go into Add/Remove Programs and see that the latest release has version 1.0.1042.
Just to make sure we don't have a bug with your JS file scenario...did you happen to edit that file in any other editor? Since you did create the original JS file in WebMatrix it should have a BOM starting out. And continuing to edit in WebMatrix should
keep the file as UTF8 BOM. If you still saw that the JS file ended up as ANSI, we can try to figure out why WebMatrix stripped out the BOM.
And thank you for your suggestions. Feel free to file any ideas that you would like to see in future versions on our Connect site:
We will take the feedback from there into consideration as we work on the next release.
Feb 03, 2011 05:13 PM|luminarious|LINK
I have the same version.
WebMatrix behaved as expected, given the circumstances. The confusion was mostly on my part, because I'm more used to seeing UTF-8 files without the BOM.
P.S. This may be a browser incompatibility (I'm using Opera v11) with the forum scripts, but why must I type the paragraph tags manually? Where should I file that feedback?
Feb 03, 2011 05:23 PM|alisonl|LINK
Ah, ok. Thanks! I'm glad we were able to get to the root of the issue. For feedback about the forums itself, you can try submitting it here:
I'm not sure if that's the right place, but it is somewhere you can give comments about the iis.net site so I think the fourms might be included.