Removing MS Word Smart quotes in PeopleSoft Pages

How to get rid of those upside down question marks.

Problem Background

When you have pages that allow for a user to copy and paste into long fields, you may later see upside down question marks or other strange non-printable characters. This only applies if your PeopleSoft database is not setup for unicode. Microsoft Word uses something called “smart quotes” that are slightly different characters for single and double quotes. If you look closely, the begin quote is angled to the right and the end quote is angled to the left. When a PeopleSoft user tries to paste in some text, the smart quotes will get pasted in to the browser window but PeopleSoft cannot handle it. At some point in the application stack, these smart quotes are getting converted to upside down question marks or some other strange looking character.

I came up with a solution on how to fix this. You have to fix it in the user’s web browser before it gets submitted to the PeopleSoft web and application server layers because it is converted there. I did some testing and the smart quotes are getting converted to non-printable before it gets to the peoplecode processor. Therefore, you can’t solve the issue with PeopleCode.

This leaves us to solve the problem with javascript. I try to avoid adding Javascript to PeopleSoft pages for many different reason that is best covered in another post. As a general statement, you should try to solve the problem using the tools that PeopleTools offers and if all else fails, then drop into Javascript. This smart quote issue is one of those exceptions where we need to use javascript.

Implementing a Fix in Javascript

The smart quote removal fix that I came up with has the following components.

1. A Javascript function that will loop through the data entered on a long field and substitute certain smart quote character codes with the standard quote characters.
2. An HTML area on the page to hold the function above.
3. A Javascript function call to add a “Change” event listener on the long comment box where the user enters their text with potential smart quotes. This is very similar to the “FieldChange PeopleCode event” conceptually but we never make a trip to the application server.
4. An HTML area on the page to hold the javascript call above.

I have created a simple page that will isolate just the pieces we need. Here is a screenshot that shows a simple page that has a long comment box and two HTML areas on the page which are tied to derived fields.

The Long comment box has a “Page Field Name” set which will show as the ID field in the HTML of the page and the javascript will use this as the field reference.

Next we have some code on the Component PostBuild PeopleCode event to set the needed javascript into the HTML Areas on the page.

  GetLevel0()(1).CHG_WORK.HTMLAREA1.Value = GetHTMLText(HTML.CHG_REMOVE_SMARTQUOTE);
  /* The "COMMENTBOX" string passed as  parameter matches the page field name that is being observed. */
  GetLevel0()(1).CHG_WORK.HTMLAREA2.Value = GetHTMLText(HTML.CHG_SMARTQUOTE_ADD_OBSERVER, "COMMENTBOX");

The contents of HTML.CHG_REMOVE_SMARTQUOTE looks like the following code. This holds the removeMSWordChars javascript function which does the smart quote removal.

  

  function removeMSWordChars(e) {
      var charactersToRemove = new Array();

      var currentCharCode, intReplacement;

      charactersToRemove[8216] = 39;
      charactersToRemove[8217] = 39;
      charactersToRemove[8220] = 34;
      charactersToRemove[8221] = 34;
      charactersToRemove[8212] = 45;

      if (this.value == undefined) {
        var el = e.srcElement;
      var str = e.srcElement.value;
    }
    else
    { var el = this;
        var str = this.value
      };

     for(c=0; c < str.length; c++) {

          var currentCharCode = str.charCodeAt(c);

          if(charactersToRemove[currentCharCode] != undefined) {

              intReplacement = charactersToRemove[currentCharCode];

              str = str.substr(0,c) + String.fromCharCode(intReplacement) + str.substr(c+1);
          }
      };

     el.value = str;
     return false;
  };

  /**
   * Cross Browser helper to addEventListener.
   *
   * @param {HTMLElement} obj The Element to attach event to.
   * @param {string} evt The event that will trigger the binded function.
   * @param {function(event)} fnc The function to bind to the element. 
   * @return {boolean} true if it was successfuly binded.
   */
   function cb_addEventListener(obj, evt, fnc) {
      // W3C model
      if (obj.addEventListener) {
          obj.addEventListener(evt, fnc, false);
          return true;
      } 
      // Microsoft model
      else if (obj.attachEvent) {
          return obj.attachEvent('on' + evt, fnc);
      }
      // Browser don't support W3C or MSFT model, go on with traditional
      else {
          evt = 'on'+evt;
          if(typeof obj[evt] === 'function'){
              // Object already has a function on traditional
              // Let's wrap it with our own function inside another function
              fnc = (function(f1,f2){
                  return function(){
                      f1.apply(this,arguments);
                      f2.apply(this,arguments);
                  }
              })(obj[evt], fnc);
          }
          obj[evt] = fnc;
          return true;
      }
      return false;
  };


  

The contents of HTML.CHG_SMARTQUOTE_ADD_OBSERVER looks like this. This piece of javascript creates the “change” observer on the long character field and will call the function to remove smart quotes each time a change occurs on that field. We have this snippet at the bottom of the page so it executes last.

  var el = document.getElementById('%BIND(:1)');
  cb_addEventListener(el, 'change', removeMSWordChars);

The end result is that every time a user changes the value of the comment box the removeMSWordChars javascript function will be called and it will substitute any smart quotes found with the non-unicode equivalent.