StdImage updated to trunk

Everything is changing on Django those days, and many people contacted me because StdImage stopped working with trunk.

Basically all major changes (except GeoDjango) affected it, such as change from newforms to forms, signal refactoring, and file storage refactoring. Now it's up to date.

Remind that django-stdimage is a Django application that provides a standardized  image field(standard name including row id, standard size, and ability to create automatic thumbnails, also standarized of course).

Firefox Add-ons (update)

Some time ago, I wrote a post in this blog about the essential Firefox extensions for developers. After some time, that post is out of date, so I changed some of those extensions, and started using some new. Here is the updated list:

Firebug: Javascript debugging, graphical check of CSS properties (moving the cursor to an area of the web, you can see the html, css properties...), loading time statistics per file, and a infinite set of option.

Web developer: Disabling CSS, Javascript, Resizing browser window to standard formats...

HTML Validator: Offline version fo the w3c validator.

SearchStatus: PageRank, and Alexa rank information.

Translating Django apps. Good practices

In this article you'll find some tips, that could be useful for avoiding problems or extra work when translating your Django application.

1. Setting up the environment

Doing some trivial changes to your project structure, can avoid you of translating many string (the ones that are already translated in Django, or in any external application).

For achieving it, my tip is to copy Django itself, and all external applications to your project path, not in a PYTHONPATH directory. It can also avoid compatibility problems, and version conflicts if you're working on several  projects. Then your project root will contain something like:

__init__.py
settings.py
urls.py
django/
transdb/
myapp/

Next step is  patching Django (while it's not included in trunk) to omit the inclusion of already translated applications into your project. Here is the patch, and you can also see #7050 for further information, or know the status.

Then, when executing ./manage.py makemessages you'll find in your project catalogs, just strings that aren't previously translated.

2. Creating string

If you don't have a correct literal creation policy, then your translator will have extra work, problems, and your translation won't be as correct as it should.

The first thing to do is write literals thinking in reusability (as  software reusability but for translations). I'll show it with some examples:

Using

{% trans 'product' %}
{% trans 'Product' %}
{% trans 'product:' %}

you'll create 3 different string in your translation. Using

{% trans 'product' %}
{{ _("product")|capfirst }}
{% trans 'product' %}:

will create just one.

Another thing to consider is that some times  you consider that a word has just one meaning, or at least you don't think that could be translated using different words. But actually, when translating it to another language it can be converted to different words depending on the context. Let's use an example.

Play football
Play the guitar

Probably for most native English  speakers play doesn't have more than a subtle difference in two sentences, but if I translate it as follows:

_("play") -> jugar

Then you'll find something like

Play football -> Jugar a  futbol (what's correct)
Play the guitar -> Jugar con la guitarra (what means "To have fun with the guitar", probably without generating any sound)

This will be avoided most times, because usually we don't translate word my word, but there are few cases where we do that, and you should consider doing something like that in that case (actually I never had to do it :)

_("play <!-- an instrument -->")
_("play <!-- a sport -->")

Then when you translate into Spanish:

"play <!-- an instrument -->" -> "tocar"
"play <!-- a sport -->" -> "jugar"

And I would also translate English into English:

"play <!-- an instrument -->" -> "play"
"play <!-- a sport -->" -> "play"

There will be infinite cases that will generate issues when translating, and it'll be impossible to control everyone. I just wanted to give some tips focused to Django applications.

3. Translating

This article isn't intended to explain how to translate (I think that there is a degree at university for it ;) . But may be you should give some tips/explanations to your translators for better results.

The first thing you should explain them is how to work with some special cases in your strings. Here you have the two mos common examples that they will found:

"This is normal text, <big>and this one is bigger</big>"
"Hello %(name)s"

Unless you explain them what it means, probably you'll find something like that in you translated string (using Spanish in the example):

"Este texto es normal, <grande>y  éste es mayor</grande>"
"Hola %(nombre)s"

Of course those translations doesn't generate the expected results, because the correct ones are:

"Este texto es normal, <big>y  éste es mayor</big>"
"Hola %(name)s"

Another thing that could be clarified, specially if your translator is involved in the Web site that is being translated (or at least knows the context where every string is used), is not to create translations more specific than the original texts.

For example, imagine that you've in your application a form for personal data, and one of the fields is called "name". Then you translate your application to Catalan, and your translator knows when translating "name", that is used as person name, and translate it as "nom propi" (first name). It will look nicer by now, while being incorrect for me, so later may be you'll add a form where you ask corporate information and you have a field "name" for the company name. You won't send the string "name" to the translator again, and your translation will be incorrect, so "nom propi" (first name) is not valid for the company name.

4. Choosing the main language

Sometimes it isn't so obvious  the language your application is written in (I mean the language you use inside gettext strings, or trans/blocktrans tags).

If you're writing an application that will be used widely in the world, and it will be translated to many languages, probably you think that English should be the right language for it, but in some cases there are some questions to take care on.

• Will your company have an international team (specially of Django developers)? If you have workers from many countries, probably English will be good for letting all of them write/read from the code.


• Will your translation team/company use English as it source language? The .po files will show your main language as source for translating string. If you hire a German to French translator, isn't a good idea writing your strings in English, so your/their work will increase a lot, and the reliability of the process will decrease.


• Are your coders fluent in English? It's more complicated (more work) to change a string from a the main language than from a translation. So if your developers can't write correct English, writing literals in their mother tongue language will save time and work.

Unluckily some times you'll have conflicts in previous questions, and you'll have to choose the lesser evil.

Django i18n status

Here you have what, from my point of view, is the status of django i18n. Comments will be very welcome, specially from people from countries with other i18n needs than mine (based on the idea that Django i18n is perfect for people in the US, here is the troubleshooting for my country, for sure more problems exists for people in for example China).

This list is part of the analysis that I'm doing to fix all those problems. If you want to participate in making Django also "The web framework for perfectionists outside the US", please contact me.

Subject		Comments
Translation (static content)	Yes	Django has an amazing translation system, easy to use, and exceptionally automated. Also it has bidi support. Despite of this, some problems can be found when translating django or applications to other languages (masculine/femenine...).
Translation (database content)	No	Django doesn't support model field translation, but it can be achieved using an external application such as TransDb, django-multilingual, django-utils translation service and i18ndynamic. As far as I know only TransDb and django-multilingual are working on Django's trunk.
Calendar customization	No	Patching Django is required to change first day of week in admin calendar (first day of week is Monday according to ISO and in many countries (most Europe, most South America, and some parts of Asia). See ticket #1061
Date format (when displaying)	Yes	Dates displayed in admin are formatted according to current locale. Also custom dates can be easily formatted using date filter (f.e. {{ my_date|date:_("DATE_FORMAT") }} ). The problem here, is that few internationalized formats are defined inside django, so using django formats you can internationalize a date with format "December 31th, 2000", but not with format "12/31/2000". It can be achieved creating date formats in your catalog.
Date format (on inputs)	No	Django allows specifying one or many input formats for a date form field (using the input_formats parameter of the form field).I think that there is no way to specify the format on the admin fields, specially because the format that generates the calendar is always the same. Anyway what it's expected is not to specify the format of a field, actually it is to get the format from the current locale.
Number format (when displaying)	No	Django has just a filter (floatformat) to format numbers, where you can specify how many decimals you want. It's very difficult to customize your number format, to the format of the current locale, and even to a fixed format.
Number format (on input)	No	Django has no way to specify if you want to enter a decimal number with comma separator (dot is always used). Of course it can't be done according to current locale.

TransDb working on trunk!

Some days ago, after the merge of qs-rf to trunk, model field i18n was in trouble, so the principal package for this approach (django-multilingual) stopped working for the latest version of trunk.

This weekend we had a TransDb sprint, to try to improve our project, and contribute to fix this situation in any way.

The result: We fixed many issues on TransDb, now the project has a more conventional structure, and a setup script, and... TransDb is now working on trunk!

For achieving it we created a new branch called oldforms.

Try it now!

StdImageField: Improved image field for Django

I'm pleased to announce a new project, django-stdimage, that provides a new image field with many improvements respect to Django's core ImageField.

Features of StdImageField:

• Saved files have standardized names (using field name and object id)


• Images can be removed


• Automatically creates a thumbnail


• Automatically resizes both image and thumbnail (with optional crop to fit exactly specified size)

Here you've an example of usage:


from django.db import models
from stdimage import StdImageField

class MyClass(models.Model):
  my_image = StdImageField(upload_to='path/to/img', blank=True, \
    size=(640, 480), thumbnail_size=(100, 100, True))

If a file called "uploaded_file.png" is uploaded for object id 34, then result will be:

• /path/to/img/my_image_34.png (with bigger possible size to fit in a 640x480 area)


• /path/to/img/my_image_34.thumbnail.png (with a exact size of 100x100, cropping if necessary)

Also it will appear a check-box for deleting when using admin.

Searching on Django Snippets

One of the key websites about Django is Django Snippets, by one of the key Django developers, James Bennett.

I've followed many of the James work, and it's awesome; but I've a complaint on Django Snippets. It's the inefficient way to find anything there. The only ways that I've found is searching using a paginated list by author, language or tag (sorted alphabetically), where you can spend a huge amount of time if the tag you search starts by Z.

Of course that I can go to Google an search using site:www.djangosnippets.org, and it would work for all indexed snippets, but I think that at least this search box should be added on the site (until anybody writes the generic search engine for Django). It would also help adding result page numbers (with links) at the end of the paginated list.

Changing language on the admin

When working on multilanguage sites, a feature that many times I missed is a direct way to change the language when you're on the admin,

With standard Django the only way that exists (as far as I know), is leaving the admin, going to the website, change the language there, and come back to the admin. Not very fast.

Today, I've created a snippet that creates a drop down menu on the admin bar (just in the main page) to change the language.

Hope it helps.

DSNP 0.9 released

I know that it was just few days ago that I released another version of DSNP, but because of it, I got a lot of feedback on it, I worked hard, and finally DSNP is stable.

For now, it'll be 0.9 because it works on Django newforms-admin branch, so 1.0 will be reached when newforms-admin will be merged into trunk.

Changeset for this version is next:

• Generated sqlite file has write permissions for all users by default


• Static files are served by Django http server on development environment (DEBUG==True)


• File admin.py created to specify admin options


• Media path has changed (now, all static files are under "media" directory) due to an issue

I hope you like it.

Unable to define my urls exactly my way (resignation statement)

I've been working with Django for a while, and it helped me to get all my web developments with nice urls. But I also wanted a nice url structure...

What it is nice for me, opposite to Django default settings, is avoiding the media prefix on my media urls, so

http://localhost/media/admin/css/login.css would be http://localhost/admin/css/login.cssTrying to emulate old website structures (used widely in php sites).

That could be nice or not, but for sure it is complicated.

The first step was to setup apache for it, a little bit more complicated than the usual setup, but possible:

<LocationMatch "/((css|js|img|swf|pdf)/|favicon.ico)">
SetHandler None
</LocationMatch>

The main problem comes when using Django development http server (started by "python manage.py runserver"), and the admin. Of course you can do that, but what is not possible, is to define the same name for your admin media path, and for the admin itself.

For example:

http://localhost/admin and http://localhost/admin/css/login.css

The reason is the Django web server, processes all requests starting with the ADMIN_MEDIA_PREFIX setting with the AdminMediaHandler, what implies with that structure that all admin requests (even the ones that aren't static files) are processed by this handler, raising an error when the request isn't for a static file. The error is next.

Permission denied: ${PYTHON_PATH}/django/contrib/admin/media/

So this is my resignation statement to do what I wanted to do initially. Now, options are:

• Don't use the Django http server (use apache even for development)


• Keep the same structure but change the admin media directory (for example from "admin" to "admin-media"


• Use the "media" (or any other name) prefix

NOTE: Current version of my project DSNP is affected by this issue. I'll solve it asap.

New DSNP version

DSNP is a simple and customizable Python script, that automatically creates a working Django project.

Project (and application) creation in Django are very openend and flexible, but sometimes is useful getting all the work done for you, specially:

• If you create many Django projects with the same structure.


• If you're new to Python, and want to see a "hello world!" application working in less than one minute.


• If you want to check your Django structure with somebody's else (me).

DSNP does exactly that, automates the process of creating projects and applications in Django. The resulting website is a simple project with a single application, ready for start creating models and templates. It's also customizable, to let everyone set their own preferences in the script, and adapt it to your desired structure.

Want to try it (in five simple steps)?

svn checkout http://dsnp.googlecode.com/svn/trunk/
python dsnp.py myproject
cd myproject
python manage.py runserver
Browse http://localhost:8000/

Easy right?

Django L10n

This Sunday, I participated in This Week in Django, and tried to give some ideas on Django localization.

Here I'll post some of the ideas of the interview (and some that I missed), for serving as reference:

How to translate your application (quick guide):

• Mark every text in your application for translation:
  
  
  
  • In models.py, views.py... convert 'my text in just one language' to _('my text to translate'). Don't forget to import _: from django.utils.translation import ugettext_lazy as _
  
  
  • In templates, convert <p>Text in english</p> to <p>{% trans 'Text in many languages' %}</p> (also this can be done with blocktrans tag)
  
  
  
  


• Go to your project path and create a directory called locale (also you can do that just for an application)


• Execute ${PATH_TO_DJANGO}/bin/make-messages -l ${LANGUAGE_CODE} (where language code is en for english, es for spanish...)


• Edit ${PROJECT_PATH}/locale/${LANGUAGE_CODE}/LC_MESSAGES/django.po and set the msgstr variables with the translation of every msgid


• Run msgfmt django.po -o django.mo (I just realized after the interview that exists a django script complie-messages.py that does that for all .po files)


• And then you have your application translated. There are some settings in settings.py that need to be set for making it work (USE_I18N = True, set LANGUAGES and LANGUAGE_CODE, and specify the django.middleware.locale.LocaleMiddleware middleware)


• Then probably you'll want to have your select input with all available languages (or something like that). For it you'll have to add (r'^i18n/', include('django.conf.urls.i18n')) to your urls.py, and from your html send a POST request to /i18n/setlang with the parameter language set to desired language code


• For more stuff, and detailed information check: http://www.djangoproject.com/documentation/i18n/

Things that IMHO should be improved in Django for a better L10n expirience:

• Move localflavors outside trunk (to avoid unnecessary translation costs). Every localflavor should come with necessary translations.


• Create locale settings (besides translations), to set decimal symbol, date and time format, first day of week... and use it automatically for current locale/language.
  
  
  
  • Check tickets #1061, #3940 and #6783 that gives different approaches to this problem.
  
  
  
  


• Create translatable CharFields and TextFields. For now django-multilingual and transdb can be used for it.


• Adding something to select the language in admin (when more than one is available).


• Haven't checked it too much, but it'll be good if urls could be translated as well.

Finally I want to thank for letting me participate in TWID to Michael Trier, who is a father, husband, software architect, entrepreneur, a great journalist, and a better person. And also to Malcolm Tredinnick, who recommended me to the show (not sure if I deserved the honour), and for his unpayable help and support on my Django work.

TransDb: Pretty much easier

Today I've released a new version of TransDb, the Django package that allows storing text at database in more than one language (using the same field).

New version is pretty much easier to use, after fixing many bugs, and avoiding the use of a filter in templates.

Now, migrating your single-language application to a multi-language one is very easy, so almost the only thing you've to do is changing your model fields (no data transformation is required, it is done automatically when you translate texts at admin). A full migration procedure is available at project page.

You can find everything at Google's project page.

Media data without media directory

It's not a big trouble, but I wanted to remove the /media/ prefix on all my media links, like...


<img alt="My Image" src="/media/img/myimage.png"/>
<link rel="stylesheet" href="/media/css/mysheet.css" type="text/css"/>


and so on.

This can be easily achieved by replacing in apache's configuration file:


<Location "/media/">
SetHandler None
</LocationMatch>


by


<LocationMatch "/((css|js|img|swf|pdf)/|favicon.ico)">
SetHandler None
</LocationMatch>


Of course you have to have all you media files in folders like css, js, img... or anything that you specify in last regular expression.

UPDATE: See this post before using this approach.

Normalize name and size images in Django (and adding thumbnails)

I wanted to assign an image to every element in a model. I wanted them to have a thumbnail, and to have normalized sizes and names. And of course, I want my application to do everything automatically.

The best method I've found is next, modifying models.py (note that PIL must be installed):

def rename_image(src, field, id):
    file_ext = os.path.splitext(src)[1].lower().replace('jpg', 'jpeg')
    dst = 'img/uploaded/work_%s/%s_%s%s' % (field, field, id, file_ext)
    return dst



def resize_image(src, dst, size):
    from PIL import Image
    image = Image.open(src)
    image.thumbnail(size, Image.ANTIALIAS)
    image.save('%s%s' % (settings.MEDIA_ROOT, dst))
    return dst



class MyModel(models.Model):
    image = models.ImageField(upload_to='img/uploaded/work_image', verbose_name=_('imagen'))
    thumbnail = models.ImageField(editable=False, upload_to='img/uploaded/work_thumbnail')
    [...]



    def save(self):
        super(MyModel, self).save()
        if self.image != rename_image(self.image, 'image', self.id):
            original_filename = self.get_image_filename()
            self.thumbnail = resize_image(original_filename, rename_image(original_filename, 'thumbnail', self.id), [100, 75])
            self.image = resize_image(original_filename, rename_image(original_filename, 'image', self.id), [640, 480])
            if os.path.exists(original_filename):
                os.remove(original_filename)
            super(MyModel, self).save()


TransDb: Django’s i18n for database

Today I've created my own code for having (in Django) fields in more than one language stored in database. There were some other packages, but none of them useful for me (as commented here).

TransDb's main goal is that is simple, for application users, application programmers, and the code itself. Some work is still missing, but there is a working version at TransDb Google Code page.

Any comment will be appreciate.

Django is “order sensitive”

Sometimes I forget that django's settings.py is a Python script, and not a plain configuration file. And forgetting it causes django to behave unexpectedly. A couple of examples that happened to are related to array sorting.

Some days ago I customized middleware classes, and after that I left on my setting.py:

MIDDLEWARE_CLASSES = (
'django.middleware.locale.LocaleMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.middleware.doc.XViewMiddleware',
'django.middleware.common.CommonMiddleware',
)

With it, LocaleMiddleware doesn't work, because it requires SessionMiddleware that isn't loaded when LocaleMiddleware is executed.

Today's issue was something similar, but with templates. I customized some admin templates, copying them to a directory loaded with filesystem loader. My settings.py looked like:

TEMPLATE_LOADERS = (
'django.template.loaders.app_directories.load_template_source',
'django.template.loaders.filesystem.load_template_source',
)

With it, loaders looked first to application template directories, including the admin ones, so overriding template was never used.

Common web features in Django

Here is a brief list of common web features, and the best way I know for achieving it in Django:

• Breadcrumbs: use {{ block.super }} for recursive link inheritance [more info]


• Back button: use {{ request.META.HTTP_REFERER }} for linking to referring URL*


• Highlight active menu option: use {{ request.path }} to know requested URL and compare it with menu options * [more info]


• Pagination: use 'django.views.generic.list_detail.object_list' generic view [more info]

* it's needed to add 'request' module to TEMPLATE_CONTEXT_PROCESSORS on settings.py

DSNP 0.11 released

DSNP is a shell script that automatically setups a new Django project, with user custom settings.

It's specially useful for users that create many Django projects following same patterns.

After a first very simple release, version 0.11 has released, that includes many validations for avoiding user errors, support for PostgresSQL as well as for MySQL, and many more improvements. See CHANGELOG for full list.

Enjoy!

Django spanish localflavor at trunk

Today has been committed to Django trunk version spanish localflavor patch.

It includes selector fields for provinces and regions, and validation form functions for postal codes, phone numbers, and nif and ccc codes.

Patch was started by Ricardo J Barrios, contributed by Rob Oggie and finished by myself, who mistakenly doesn't appear as an author of it. :)

I hope you like it, and comments always will be very welcome.