6

Update (on top because the post is so long)

Alright. New developments. When I add

resources.db.params.charset = "utf8"
resources.db.params.driver_options.1002 = "SET NAMES utf8;"

to my bootstrap, the title is returned as oriëntatie (It's in the database as oriëntatie). But when I want to add oriëntatie to my DB, it gets stripped all the way down to ori.

Original post

I have built a CMS. Like any other CMS you can add/update pages.

Now when I add a page with the a title containing, for example, the character ë it gets placed in the database as such in my title field. (I use StringTrim and StripTags filters on my form.)

Now, when I want to update the page, and pre-fill the title input field I want to show the ë character again. Instead my input field remains empty.

I tried to encode and decode the value (htmletities, html_entity_decode) all possible ways, but I can only get the htmlentity value displaying in the form field.

My guts are telling me that this isn't the right way, but still I want people to add proper titles without spelling mistakes...

Advice, tips anything would be greatly appreciated!

Edit: Added some code, not sure which parts

This is what happens with the code below:

Adding the word oriëntatie through the input field puts oriëntatie in the database. While trying to load the value oriëntatie in the input field again on the update page, the input field stays empty. I now for sure that all the data is retrieved.

Below is a screenshot of the database row filled.

enter image description here

Application.ini

resources.db.adapter = PDO_MYSQL
resources.db.params.host = localhost
resources.db.params.profiler = true

Bootstrap

// Build the view and layouts
protected function _initBuildBase()
{
    $this->bootstrap('view');
    $this->bootstrap('layout');     
    $layout = $this->getResource('layout');
    $this->view = $layout->getView();

    $this->view->doctype("HTML4_STRICT");
    $this->view->setEncoding('UTF-8');
    $this->view->headMeta()->appendHttpEquiv('Content-Type', 'text/html;charset=utf-8');
    $this->view->headMeta()->appendHttpEquiv('Content-Language', 'nl-NL');
    $this->view->headMeta()->appendHttpEquiv('Cache-control', 'public');
    $this->view->headMeta()->appendName('author', 'De Graaf & Partners Communications');
}

Header of the update.phtml page

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">

    <head>

        <link href="/server_management/domains/cms_version_2/../../_application/public/images/admin/favicon.ico" rel="icon" type="image/x-icon" /><meta http-equiv="Content-Type" content="text/html;charset=utf-8" >

<meta http-equiv="Content-Language" content="nl-NL" >

<meta http-equiv="Cache-control" content="public" >

<meta name="author" content="De Graaf &amp; Partners Communications" >

<meta name="robots" content="noindex, nofollow" ><link href="/server_management/domains/cms_version_2/../../_application/public/css/admin/style.css" media="screen" rel="stylesheet" type="text/css" >

<!--[if IE]> <link href="/server_management/domains/cms_version_2/../../_application/public/css/admin/ie/style.css" media="screen, projection" rel="stylesheet" type="text/css" ><![endif]-->

<!--[if IE]> <link href="/server_management/domains/cms_version_2/../../_application/public/css/blueprint/ie.css" media="screen, projection" rel="stylesheet" type="text/css" ><![endif]-->

<link href="/server_management/domains/cms_version_2/../../_application/public/css/admin/print.css" media="print" rel="stylesheet" type="text/css" ><script type="text/javascript" src="/server_management/domains/cms_version_2/../../_application/public/jquery/jquery.lib.js"></script>

<script type="text/javascript" src="/server_management/domains/cms_version_2/../../_application/public/jquery/jquery.loader.js"></script>

<script type="text/javascript" src="/server_management/domains/cms_version_2/../../_application/public/jquery/jquery.init.js"></script>

<script type="text/javascript" src="/server_management/domains/cms_version_2/../../_application/public/jquery/tinymce/jquery.tinymce.js"></script><title>Pages - Admin - DGPCMS</title> </head>

    <body>

Database

enter image description here

Database Table

enter image description here

PagesService (Insert and Update)

public function InsertPages($url, $parent_page, $title, $text, $keywords, $description, $user, $dashboardmessage)
    {
        $data = array(
            'url' => $url,
            'parent_page' => $parent_page,
            'secure' => 'n',
            'title' => $title,
            'text' => $text,
            'keywords' => $keywords,
            'description' => $description,
            'user_created' => $user,
            'user_modified' => $user,
            'date_created' => time(),
            'date_modified' => time()
        );
        return $this->pages->insert($data);
        $this->DashboardService->InsertDashboard('insert', 'pages', $dashboardmessage, $user);
    }

    public function UpdatePages($id, $url, $parent_page, $title, $text, $keywords, $description, $user, $dashboardmessage)
    {
        $data = array(
            'url' => $url,
            'parent_page' => $parent_page,
            'secure' => 'n',
            'title' => $title,
            'text' => $text,
            'keywords' => $keywords,
            'description' => $description,
            'user_modified' => $user,
            'date_modified' => time()
        );
        $this->pages->update($data, $this->CreateWhereClause($id));
        $this->DashboardService->InsertDashboard('update', 'pages', $dashboardmessage, $user);
    }

PagesController (preDispatch, form settings)

$this->view->form = new Forms_Pages();
$this->view->form->setElementFilters(array('StringTrim', 'StripTags'));
$this->view->standardform = new Forms_StandardButtons();
$this->view->standardform->setElementFilters(array('StringTrim', 'StripTags'));

PagesController (Insert and Update)

public function insertAction()
{
    $this->view->pagesDropdown($this->PagesService->GetAllRootPages(), 'url');
    $pass = false;
    $textArray = array();
    foreach($this->PagesService->GetAllPages() as $result)
    {
        $textArray[] = $result->text;
    }
    if($this->getRequest()->isPost())
    {
        if($this->view->form->isValid($this->getRequest()->getPost()))
        {
            if($this->checkexists->isValid($this->view->urlCleaner($this->view->form->getValue('title'))))
            {
                if(preg_match('/(\\[\\[news:overview\\]\\])/is', $this->view->form->getUnfilteredValue('text')))
                {
                    if(preg_grep('/(\\[\\[news:overview\\]\\])/is', $textArray))
                    {
                        $this->_helper->flashMessenger(array('message' => $this->view->translate('The tag [[news:overview]] was already placed on another page. Please remove it before placing it on another page'), 'status' => 'notice'));
                    }
                    else
                    {
                        $pass = true;
                        $this->cache->save($this->view->urlCleaner($this->view->form->getValue('title')), 'module_newsBasepage');
                    }
                }
                else
                {
                    $pass = true;
                }
                if($pass)
                {
                    $this->lastId = $this->PagesService->InsertPages(
                        $this->view->urlCleaner($this->view->form->getValue('title')),
                        $this->view->form->getValue('parent_page'),
                        $this->view->form->getValue('title'),
                        stripslashes($this->view->form->getUnfilteredValue('text')),    
                        $this->view->form->getValue('keywords'),    
                        $this->view->form->getValue('description'),
                        $this->view->user->username,
                        '<strong>'.$this->view->form->getValue('title').'</strong>'
                    );
                    $this->_helper->flashMessenger(array('message' => $this->view->translate('The '.$this->view->subject.' was succesfully saved'), 'status' => 'success'));
                    if($this->getRequest()->getPost('save_finish') != 'Save')
                    {
                        $this->_redirect('/admin/pages/update/'.$this->lastId);
                    }
                    else
                    {
                        $this->_helper->redirectToIndex();
                    }
                }
            }
            else
            {
                $this->_helper->flashMessenger(array('message' => $this->view->translate('This '.$this->view->subject.' already exists'), 'status' => 'notice'));
            }
        }
        else
        {
            $this->_helper->flashMessenger(array('message' => $this->view->translate('Some errors occured'), 'status' => 'error'));
        }
    }
}

public function updateAction()
{
    $this->view->result = $this->PagesService->GetSinglePage($this->_getParam('id'));
    $this->view->form->populate($this->view->result[0]);
    //$this->view->form->populate(array('title' => html_entity_decode($this->view->result[0]['title'])));
    $this->view->pagesDropdown($this->PagesService->GetAllRootPages(), 'url', $this->view->result[0]['title']);
    $pass = false;
    $textArray = array();
    if($this->getRequest()->isPost())
    {
        if($this->view->form->isValid($this->getRequest()->getPost()))
        {
            foreach($this->PagesService->GetAllPages() as $result)
            {
                if($result->id != $this->view->result[0]['id'])
                {
                    $textArray[] = $result->text;
                }
            }
            if($this->view->form->getValue('title') != $this->view->result[0]['title'])
            {
                if($this->checkexists->isValid($this->view->urlCleaner($this->view->form->getValue('title'))))
                {
                    $pass = true;
                }
                else
                {
                    $this->_helper->flashMessenger(array('message' => $this->view->translate('This '.$this->view->subject.' already exists'), 'status' => 'notice'));
                }
            }
            if(preg_match('/(\\[\\[news:overview\\]\\])/is', $this->view->form->getUnfilteredValue('text')))
            {
                if(preg_grep('/(\\[\\[news:overview\\]\\])/is', $textArray))
                {
                    $this->_helper->flashMessenger(array('message' => $this->view->translate('The tag [[news:overview]] was already placed on another page. Please remove it before placing it on another page'), 'status' => 'notice'));
                }
                else
                {
                    $pass = true;
                    $this->cache->save($this->view->urlCleaner($this->view->form->getValue('title')), 'module_newsBasepage');
                }
            }
            else
            {
                $pass = true;
            }
            if($pass == true)
            {
                $this->lastId = $this->PagesService->UpdatePages(
                    $this->_getParam('id'),
                    $this->view->urlCleaner($this->view->form->getValue('title')),
                    $this->view->form->getValue('parent_page'),
                    $this->view->form->getValue('title'),
                    stripslashes($this->view->form->getUnfilteredValue('text')),    
                    $this->view->form->getValue('keywords'),    
                    $this->view->form->getValue('description'),
                    $this->view->user->username,
                    '<strong>'.$this->view->form->getValue('title').'</strong>'
                );
                $this->_helper->flashMessenger(array('message' => $this->view->translate('The '.$this->view->subject.' was succesfully saved'), 'status' => 'success'));
                if(!$this->getRequest()->getPost('save_finish') != 'Save')
                {
                    $this->_helper->redirectToIndex();
                }
                else
                {
                    $this->_redirect('/admin/pages/update/'.$this->_getParam('id'));
                }
            }
        }
        else
        {
            $this->_helper->flashMessenger(array('message' => $this->view->translate('Some errors occured'), 'status' => 'error'));
        }
    }
}

This is the result of Zend_Debug::dump($this->view->result);

array(1) {
  [0] => array(13) {
    ["id"] => string(3) "188"
    ["order"] => string(1) "0"
    ["url"] => string(10) "orientatie"
    ["parent_page"] => string(3) "n/a"
    ["secure"] => string(1) "n"
    ["title"] => string(10) "oriëntatie"
    ["text"] => string(13) "<p>Test 3</p>"
    ["keywords"] => string(6) "Test 1"
    ["description"] => string(6) "Test 2"
    ["user_created"] => string(5) "Admin"
    ["user_modified"] => string(5) "Admin"
    ["date_created"] => string(10) "1326280122"
    ["date_modified"] => string(10) "1326280122"

And this is the html output

<div class="padding_row">
<label for="title" class="required">Title</label>
<input type="text" name="title" id="title" value="" class="form_validator"> <div class="form_validator_box"> <a href="#" title="This page already exists" class="form_validator_result_bad"></a> </div>
</div>
Improve this question
11
  • What charset are you using in HTML? Make sure to use UTF-8 or something similar that supports characters you need. Also make sure your files are saved as UTF-8, and that your tables in the database are using UTF-8 encoding. Also don't forget to SET NAMES to UTF-8 for your db connection. Commented Jan 10, 2012 at 13:46
  • I have set the HTML charset to UTF-8 in my bootstrap and my database collation is utf8_general_ci. I didn't had the SET NAMES, so I added resources.db.params.charset = utf8 to my config file. Now it wil break the string: Without the rule in the config file the word 'oriëntatie' was fine, but now it inserts 'ori' in the database. Commented Jan 10, 2012 at 14:13
  • 1
    Try posting some of your code as is we are taking wild guesses. Commented Jan 10, 2012 at 16:27
  • Can you see oriëntatie properly in phpMyAdmin? Commented Jan 11, 2012 at 9:14
  • How exactly do you populate the form? Commented Jan 11, 2012 at 9:16

5 Answers 5

Reset to default
1

The only time I've seen this happen is when the client (browser) isn't aware of the correct character encoding.

I see you've added the appropriate HTTP-Equiv meta into the HeadMeta helper but are you actually displaying it in your view or layout?

You should have something like this in the <head> section of your layout (or view if not using a layout)

<head>
    <?php echo $this->headMeta() ?>

I'm a little curious about your Bootstrap class. Could you show the rest of the code around the two lines you've listed? Why is the view apparently a property of Bootstrap and how is it assigned?

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

I updated the bootstrap code (put on the whole function) and I put on the header of the html page. If you need more info let me know.
1

The dump shows the correct string. So this cannot be a database issue.

But I noticed, that you do 

$this->view->form->isValid($this->getRequest()->getPost())

AFTER populate() in your update action. The isValid method includes populating the form already. So here you might overwrite your string with an empty string from POST. You should move your populate() to 

if($this->getRequest()->isPost()) {
    //...
}
else{
    // here
}

or remove it and replace isValid(...) with

$this->view->form->isValid($this->view->result[0])

If the error is still somewhere else insert 

Zend_Debug::dump($this->view->form->getValue('title'));

after every major operation to determine where your string changes to blank.

Improve this answer

5 Comments

But I want to repopulate the fields on the page load. Then when the user submits the form I want to check the last data entered (changes) by the user. Removing all the code (except for the populating rule) and the do a dump on the title field it outputs the right string. It's just not displaying in the input field for some reason. I think it's an utf 8 thing. But thanks anyway :)
Does it work with a string without special chars? If so, you are right, then it's probably utf 8 related.
works perfect with strings without special chars. Any ideas on the utf8?
So what db connection configuration are you using right now? With SET NAMES UTF-8 or without? Have you tried re-inserting the problematic row after adding SET NAMES in the cfg?
I'm using it without the 'set names' for now. I tried adding it again after adding the 'set names'. Like it says in my OP it will strip the string from the special char and on; 'oriëntatie' will be entered in the database as 'ori' with the 'set names' added in the cfg
0

Since the HTML page contains the Content-Type text/html;charset=utf-8 and the database has the correct collation and charset then the problem must be in the application part.

Next possible problem could be the connection to the database, but since you have set "SET NAMES 'utf8'" upon mySQL connection then the only logical next step it that the problem might be on the:

$this->view->form->setElementFilters(array('StringTrim', 'StripTags'));

Could you try removing the StripTags from the filters so that you can test if the problem is there?

Improve this answer

7 Comments

I tried that, but it has no effect. I removed the "set names" because it will strip everything from the special char on (oriëntatie becomes ori in the database). For testing purposes I did add it again, but it made no difference. Thanks anyway!
@RickdeGraaf I would suggest to print_r() the values at each point along the way to the database. Start by doing a print_r($_POST) to verify that they have been correctly received from the browser, then print_r($form->getValues()); to verify the form's output, and once more before entering them to the database. You are definitely going to find where there is the problem. Also the set names should be included if you are passing UTF-8 data. The problem is not there. I've written tons of UTF-8 web apps and SET NAMES 'utf8' is a must
Oke, I started to look on where it is entered wrongly in the database. I found out that this is not in my controller. It happens in my service, but Zend somewhere in the db class removes all special chars, because the value I pass to the service is good. If I return the values send to insert ($db->insert(array(values))) the value is still good. This does not happen when I remove the set names rule from my config. What could be the cause here?
@RickdeGraaf Just to be safe, through the Zend_Db Profiler can you retrieve the final Insert SQL statement sent to the database? If Zend_Db->insert() is causing the problem you should see the values corrupted. Btw I can see a few errors in your PagesService class. You return from the InsertPages() function but after that you have another call (which never gets executed) and on the UpdatePages() you never return anything back (but you seem to expect some return on the controller part)
Yeah, the problem with the return rule I already fixed somewhere between posting this and now. And I copied that bit for the updateAction. It don't need an return, forgot to remove the var after the copy/paste from insert. Thanks for the heads up. I added the database profile to the page. It returns this: INSERT INTO system_pages (url, parent_page, secure, title, text, keywords, description, user_created, user_modified, date_created, date_modified) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
0

Could it be you are using this bug ? http://framework.zend.com/issues/browse/ZF-11533

Improve this answer

6 Comments

I updated my library to the latest version of the framework, but it madee no difference
see i have had this same problem, the bug is present even in the latest version 1.11.11, try to change that line as it says in the bug, should fix your problem
I did that, but it returns my value as empty, the validator is always triggered and saying that the field was empty (while it wasn't)
what php version you have? just adding that "u" should make it unicode, not drop the whole string, can you double check you modified it right?
Im running PHP Version 5.3.5. I'm absolutly sure I modified the rule (only adding a u). I did it again to be sure, but still the validator gets triggered with or without input.
|
0

I found the solution, and I can beat the crap out of myself for not thinking of this sooner and missing this completely.

First the solution:

In my .htaccess file i had the rule AddDefaultCharset iso-8859-1 which off course had to be AddDefaultCharset UTF-8

After a few days I decided to review the problem after leaving it alone for a few days (though it bugged me throughout these days). I looked up how to set the character encoding for a Zend application. All my settings were correct, accept I didn't had the accept-charset set to my form. So After I did this, I got a return: orïëntätïës.

Data was added in the database correctly, so I figured I had to utf-8_decode the return and I was right. Now I knew for sure that somewhere a different charset was set. I checked the description of utf8_decode on php.net (link) and this triggered me:

utf8_decode — Converts a string with ISO-8859-1 characters encoded with UTF-8 to single-byte ISO-8859-1

Then I did a search through my complete application for the string ISO-8859-1 and the search returned that this was in my .htaccess file. So all I had to do is change that, and all was fine :)

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.