Friday, January 27, 2012

Translation Web Service in C#

Simple, and can be broken down into the following subtasks:
  1. Get text for translation and encode it into a HTTP POST request
  2. Send the data to the web server, acting in effect as a .NET web browser
  3. Read the response back into a big string
  4. Remove all the HTML and formatting and send the raw translated string back to the client.
So fire up Visual Studio .NET, and create an ASP.NET Web Service, and name it Translation, and add aTranslate.asmx file. There are two inputs: the translation mode (e.g., French to English), and the data to betranslated (e.g., 'the quick brown fox jumps over the lazy dog'). To make it a plug-in replacement for the oldservice, I gave my method the same name and parameters as the old one:
public string BabelFish(string translationmode, string sourcedata) 
The translation modes can be found in the source of the page at Babelfish:
readonly string[] VALIDTRANSLATIONMODES = new string[] 
 {"en_zh", "en_fr", "en_de", "en_it", "en_ja", "en_ko", "en_pt", "en_es", 
 "zh_en", "fr_en", "fr_de", "de_en", "de_fr", "it_en", "ja_en", "ko_en", 
 "pt_en", "ru_en", "es_en"};
The code performs validation to check for a valid mode before passing it on to Babelfish. After that, we create a POST request. The syntax for a HTTP POST request looks something like this:
POST /babelfish/tr/ HTTP/1.0
Content-Type: application/x-www-form-urlencoded
Content-Length: 51

It's pretty simple, and if you want, you could use low-level sockets to write the data to the server. Microsoft provides some better ways to do this however, and so we use the HttpWebRequest class, which has lots of built-in features to make it easy to work with HTTP connections.
Uri uri = new Uri(BABELFISHURL);
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(uri);
request.Referer = BABELFISHREFERER;
// Encode all the sourcedata 
string postsourcedata;
postsourcedata = "lp=" + translationmode + 
    "&tt=urltext&intl=1&doit=done&urltext=" + 
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
request.ContentLength = postsourcedata.Length;
request.UserAgent = "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)";
Stream writeStream = request.GetRequestStream();
UTF8Encoding encoding = new UTF8Encoding();
byte[] bytes = encoding.GetBytes(postsourcedata);
writeStream.Write(bytes, 0, bytes.Length);
HttpWebResponse response = (HttpWebResponse) request.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader readStream = new StreamReader (responseStream, Encoding.UTF8);
string page = readStream.ReadToEnd();
We end up with a string containing the entire Babelfish page. As it stands, this is about 99% noise (HTML tags, Altavista information, etc.), and 1% the translation we were looking for. So we need a regular expression to find the translatetext. By looking at the HTML page, you will find the translation is contained between:
<Div style=padding:10px; lang=fr>translation here</div>
So the required regular expression looks like this (note: while testing my regular expressions, I got lots of help from Regulator):
<Div style=padding:10px; lang=..>((?:.|\n)*?)</div>
This will match the whole <div>...</div> string. This is a fairly complex regular expression, but basically, the. character matches everything, except for newlines, hence the (.|\n) pattern, which means any character (except newlines) or new lines.
The brackets create a matching group, meaning that the text within the brackets (namely the translation) will be put in its own group at index 1 (index 0 contains the whole match).
The ?: pattern suppresses grouping: () normally creates a matching group: in this case, we are only using the pattern to allow for line breaks in long translations.
Finally *? is a lazy regular expression, matching every character up to the first instance of <div>. (If I had used plain *, the expression would be greedy, and would chomp right up to the LAST </div>.)
Here's the code:
Regex reg = new Regex(@"<Div style=padding:10px; lang=..>(.*?)</div>");
MatchCollection matches = reg.Matches(page);
if (matches.Count != 1 || matches[0].Groups.Count != 2) 
    return ERRORSTRINGSTART + "The HTML returned from Babelfish " + 
        "appears to have changed. Please check for" + 
        " an updated regular expression" + 
return matches[0].Groups[1].Value;
And subject to error checking, that's it!

Using it

Download the code, and unzip it somewhere. Add a virtual directory called Translation in IIS. Go to/translate.asmx and click Test, and enter some test data (say 'en_fr', and 'cheese'). If it works, you are ready to use it in your web and Windows Forms applications.
To use it in your app, add a Web Reference to the asmx, to the program you want to use it in; Visual Studio will create a proxy reference for you, which you can then use to perform translation.
Here's some sample code-behind:
namespace test
    using System;
    using System.Data;
    using System.Drawing;
    using System.Web;
    using System.Web.UI.WebControls;
    using localhost1; // assuming that's the reference generated
    using System.Web.UI.HtmlControls;

    /// <summary>
    ///     Summary description for WebUserControl1.
    /// </summary>
    public class WebUserControl1 : System.Web.UI.UserControl
        protected System.Web.UI.WebControls.DropDownList ddTranslationMode;
        protected System.Web.UI.WebControls.TextBox txtText;
        protected System.Web.UI.WebControls.Label lblTranslation;
        protected System.Web.UI.WebControls.Button submitButton;

        private void Page_Load(object sender, System.EventArgs e)
            // Put user code to initialize the page here

        protected void submitButton_Click(object sender, System.EventArgs e) 
            string translationMode = 
            string translationText = this.txtText.Text.Trim();
            string translation = "";
                Translate tr = new Translate();
                translation = tr.BabelFish(translationMode,translationText);
            catch (Exception exp) 
                translation = "There was an error accessing the server: " 
                                                             + exp.Message;
            this.lblTranslation.Text = translation;

No comments: