Showing posts with label IT. Show all posts
Showing posts with label IT. Show all posts

Friday, January 1, 2010

Linux and Debian simple boot

Today I've been researching on Linux and Debian booting.

There is an excellent article from IBM, which explains the procedure, and the involved parts:

http://www.ibm.com/developerworks/linux/library/l-linuxboot/

Basically:
  • BIOS checks CMOS and choose the booting device
  • Control is given to device's MBR (physically first 512 bytes)
  • MBR checks for partitions on the device (in a self contained table), and gives control to bootable partition.
  • Then, Grub, LILO or whatever takes control, to load the kernel, and the file system.
  • Usually a initrd filesystem is loaded before the "real" one. This way, the kernel can access this filesystem, while the modules for loading the one in the root partition are not yet loaded.
  • Finally, init program is called, to load all user-space applications.
To set this up in a USB drive (my idea), in a simple way, we need:

Make the drive bootable, using the syslinux tool, which is used for FAT filesystems:
syslinux /dev/sdb (or whatever device you want)

Then, mount the filesystem, and copy:
linux: the linux kernel binary
initrd.gz: compressed cpio file containing the initrd file tree

and

syslinux.cfg: syslinux settings, to let syslinux know where to find the kernel and the initrd. Basically:

default linux
append initrd=initrd.gz

Then, just restart, and your device will boot your kernel, and your filesystem.

Here, you can find a Linux kernel, and a initrd file, which will load a basic linux system, running the Debian installer:


Some more info on it at:


Thursday, August 28, 2008

Compare two XML strings in Python

I had to compare two XML strings for some unit tests, and if you want to do it without considering the indentation, or the newlines, it is a little bit tricky.

I thought that parsing the original xml and returning it again (using minidom), I'd got a raw string without any meaningless space, or any newline, but actually it returned the original string. Using toprettyxml() method also returns a trivial result, based on the original string (even when you specify the indent and the newline characters).

So the best way I've found by now is to write a custom function that returns what I want, an XML string without any trivial character between tag and tag. Here you have the code:

def raw_xml(xml_str):
    from xml.dom import minidom
    xml = minidom.parseString(xml_str)
    return u''.join([unicode(line).strip() for line in xml.toprettyxml().splitlines()])

Sunday, August 17, 2008

Brother printer and GNU (aka Linux)

Some time ago I purchased a Brother multifunctional priner/scanner/fax, the MFC-235C.

Today, after reinstalling my laptop, I had to install it again, and it has been as easy as the first. Specially because Brother has GNU (aka Linux) support, including GPL drivers for all printer features.

As well as the deserved publicity that I want to give to Brother in this post, I want to write a quick installation howto. I'm installing the printer locally using USB, and CUPS in my Debian. You should have this installed in your computer before starting the next steps (otherwise you'll have to add some steps, or change some).

And that's it! Very easy compared with the problems that I had doing the same installation for HP all-in-one printers.

Wednesday, August 13, 2008

StdImage updated to trunk

Everything is changing on Django those days, and many people contacted me because StdImage stopped working with trunk.

Basically all major changes (except GeoDjango) affected it, such as change from newforms to forms, signal refactoring, and file storage refactoring. Now it's up to date.

Remind that django-stdimage is a Django application that provides a standardized  image field(standard name including row id, standard size, and ability to create automatic thumbnails, also standarized of course).

Friday, August 8, 2008

Cuil? WTF!

Few days ago a new search engine was published, and many media covered the it. I supposed for that, that it should be something important, but I think that in few days it'll only be important for its creators. Why I think that? Well, here you have my arguments for that conclusion.

From Cuil: "Cuil searches more pages on the Web than anyone else—three times as many as Google and ten times as many as Microsoft."

From Google: "when our systems that process links on the web to find new content hit a milestone: 1 trillion (as in 1,000,000,000,000) unique URLs on the web at once!"

From myself:  1,000,000,000,000 * 3 = 3,000,000,000,000

From Alexa: "Vaig.be has a traffic rank of 1,163,358"

Conclusion:  You're supposed to have 1 trillion pages, and you don't have my blog, that is the 1 million page more visited around the world? WTF!

Thursday, August 7, 2008

Firefox Add-ons (update)

Some time ago, I wrote a post in this blog about the essential Firefox extensions for developers. After some time, that post is out of date, so I changed some of those extensions, and started using some new. Here is the updated list:

Firebug: Javascript debugging, graphical check of CSS properties (moving the cursor to an area of the web, you can see the html, css properties...), loading time statistics per file, and a infinite set of option.

Web developer: Disabling CSS, Javascript, Resizing browser window to standard formats...

HTML Validator: Offline version fo the w3c validator.

SearchStatus: PageRank, and Alexa rank information.

Friday, August 1, 2008

Translating Django apps. Good practices

In this article you'll find some tips, that could be useful for avoiding problems or extra work when translating your Django application.

1. Setting up the environment

Doing some trivial changes to your project structure, can avoid you of translating many string (the ones that are already translated in Django, or in any external application).

For achieving it, my tip is to copy Django itself, and all external applications to your project path, not in a PYTHONPATH directory. It can also avoid compatibility problems, and version conflicts if you're working on several  projects. Then your project root will contain something like:

__init__.py
settings.py
urls.py
django/
transdb/
myapp/


Next step is  patching Django (while it's not included in trunk) to omit the inclusion of already translated applications into your project. Here is the patch, and you can also see #7050 for further information, or know the status.

Then, when executing ./manage.py makemessages you'll find in your project catalogs, just strings that aren't previously translated.

2. Creating string

If you don't have a correct literal creation policy, then your translator will have extra work, problems, and your translation won't be as correct as it should.

The first thing to do is write literals thinking in reusability (as  software reusability but for translations). I'll show it with some examples:

Using

{% trans 'product' %}
{% trans 'Product' %}
{% trans 'product:' %}

you'll create 3 different string in your translation. Using

{% trans 'product' %}
{{ _("product")|capfirst }}
{% trans 'product' %}:

will create just one.

Another thing to consider is that some times  you consider that a word has just one meaning, or at least you don't think that could be translated using different words. But actually, when translating it to another language it can be converted to different words depending on the context. Let's use an example.

Play football
Play the guitar


Probably for most native English  speakers play doesn't have more than a subtle difference in two sentences, but if I translate it as follows:

_("play") -> jugar


Then you'll find something like

Play football -> Jugar a  futbol (what's correct)
Play the guitar -> Jugar con la guitarra (what means "To have fun with the guitar", probably without generating any sound)


This will be avoided most times, because usually we don't translate word my word, but there are few cases where we do that, and you should consider doing something like that in that case (actually I never had to do it :)

_("play <!-- an instrument -->")
_("play <!-- a sport -->")


Then when you translate into Spanish:

"play <!-- an instrument -->" -> "tocar"
"play <!-- a sport -->" -> "jugar"


And I would also translate English into English:

"play <!-- an instrument -->" -> "play"
"play <!-- a sport -->" -> "play"


There will be infinite cases that will generate issues when translating, and it'll be impossible to control everyone. I just wanted to give some tips focused to Django applications.

3. Translating

This article isn't intended to explain how to translate (I think that there is a degree at university for it ;) . But may be you should give some tips/explanations to your translators for better results.

The first thing you should explain them is how to work with some special cases in your strings. Here you have the two mos common examples that they will found:

"This is normal text, <big>and this one is bigger</big>"
"Hello %(name)s"


Unless you explain them what it means, probably you'll find something like that in you translated string (using Spanish in the example):

"Este texto es normal, <grande>y  éste es mayor</grande>"
"Hola %(nombre)s"


Of course those translations doesn't generate the expected results, because the correct ones are:

"Este texto es normal, <big>y  éste es mayor</big>"
"Hola %(name)s"


Another thing that could be clarified, specially if your translator is involved in the Web site that is being translated (or at least knows the context where every string is used), is not to create translations more specific than the original texts.

For example, imagine that you've in your application a form for personal data, and one of the fields is called "name". Then you translate your application to Catalan, and your translator knows when translating "name", that is used as person name, and translate it as "nom propi" (first name). It will look nicer by now, while being incorrect for me, so later may be you'll add a form where you ask corporate information and you have a field "name" for the company name. You won't send the string "name" to the translator again, and your translation will be incorrect, so "nom propi" (first name) is not valid for the company name.

4. Choosing the main language

Sometimes it isn't so obvious  the language your application is written in (I mean the language you use inside gettext strings, or trans/blocktrans tags).

If you're writing an application that will be used widely in the world, and it will be translated to many languages, probably you think that English should be the right language for it, but in some cases there are some questions to take care on.

  • Will your company have an international team (specially of Django developers)? If you have workers from many countries, probably English will be good for letting all of them write/read from the code.

  • Will your translation team/company use English as it source language? The .po files will show your main language as source for translating string. If you hire a German to French translator, isn't a good idea writing your strings in English, so your/their work will increase a lot, and the reliability of the process will decrease.

  • Are your coders fluent in English? It's more complicated (more work) to change a string from a the main language than from a translation. So if your developers can't write correct English, writing literals in their mother tongue language will save time and work.


Unluckily some times you'll have conflicts in previous questions, and you'll have to choose the lesser evil.

Saturday, July 26, 2008

Django i18n status

Here you have what, from my point of view, is the status of django i18n. Comments will be very welcome, specially from people from countries with other i18n needs than mine (based on the idea that Django i18n is perfect for people in the US, here is the troubleshooting for my country, for sure more problems exists for people in for example China).


This list is part of the analysis that I'm doing to fix all those problems. If you want to participate in making Django also "The web framework for perfectionists outside the US", please contact me.



















































Subject
Comments
Translation (static content)YesDjango has an amazing translation system, easy to use, and exceptionally automated. Also it has bidi support. Despite of this, some problems can be found when translating django or applications to other languages (masculine/femenine...).
Translation (database content)NoDjango doesn't support model field translation, but it can be achieved using an external application such as TransDb, django-multilingual, django-utils translation service and i18ndynamic. As far as I know only TransDb and django-multilingual are working on Django's trunk.
Calendar customizationNoPatching Django is required to change first day of week in admin calendar (first day of week is Monday according to ISO and in many countries (most Europe, most South America, and some parts of Asia). See ticket #1061
Date format (when displaying)YesDates displayed in admin are formatted according to current locale. Also custom dates can be easily formatted using date filter (f.e. {{ my_date|date:_("DATE_FORMAT") }} ).

The problem here, is that few internationalized formats are defined inside django, so using django formats you can internationalize a date with format "December 31th, 2000", but not with format "12/31/2000". It can be achieved creating date formats in your catalog.

Date format (on inputs)NoDjango allows specifying one or many input formats for a date form field (using the input_formats parameter of the form field).I think that there is no way to specify the format on the admin fields, specially because the format that generates the calendar is always the same.

Anyway what it's expected is not to specify the format of a field, actually it is to get the format from the current locale.

Number format (when displaying)NoDjango has just a filter (floatformat) to format numbers, where you can specify how many decimals you want. It's very difficult to customize your number format, to the format of the current locale, and even to a fixed format.
Number format (on input)NoDjango has no way to specify if you want to enter a decimal number with comma separator (dot is always used). Of course it can't be done according to current locale.

Monday, May 19, 2008

TransDb working on trunk!

Some days ago, after the merge of qs-rf to trunk, model field i18n was in trouble, so the principal package for this approach (django-multilingual) stopped working for the latest version of trunk.

This weekend we had a TransDb sprint, to try to improve our project, and contribute to fix this situation in any way.

The result: We fixed many issues on TransDb, now the project has a more conventional structure, and a setup script, and... TransDb is now working on trunk!

For achieving it we created a new branch called oldforms.

Try it now!

Saturday, May 17, 2008

StdImageField: Improved image field for Django

I'm pleased to announce a new project, django-stdimage, that provides a new image field with many improvements respect to Django's core ImageField.

Features of StdImageField:

  • Saved files have standardized names (using field name and object id)

  • Images can be removed

  • Automatically creates a thumbnail

  • Automatically resizes both image and thumbnail (with optional crop to fit exactly specified size)


Here you've an example of usage:


from django.db import models
from stdimage import StdImageField


class MyClass(models.Model):
  my_image = StdImageField(upload_to='path/to/img', blank=True, \
    size=(640, 480), thumbnail_size=(100, 100, True))


If a file called "uploaded_file.png" is uploaded for object id 34, then result will be:

  • /path/to/img/my_image_34.png (with bigger possible size to fit in a 640x480 area)

  • /path/to/img/my_image_34.thumbnail.png (with a exact size of 100x100, cropping if necessary)


Also it will appear a check-box for deleting when using admin.

Sunday, May 11, 2008

Adding your Google Code project to Google Analytics

I know that the post title looks like a dummy tutorial, but actually it's more a bug report.

In the past I had no problem setting up the Google Analytics tracker for my Google code projects, but I tried it today and it's more complicate due to what like looks to be a bug.

The problem is when creating a new profile and specifying something like code.google.com/p/myproject as the URL to be tracked, because now, just  domain names without any path are allowed, and n "Invalid input" error is raised.

The solution is simple if you know it. You can create your profile just with the domain name (code.google.com), and then edit the profile and update profile name and profile url to the correct values (in this page it's allowed).

I guess that it will be quickly  solved by the Google team, but until then, here you have the tip.

Searching on Django Snippets

One of the key websites about Django is Django Snippets, by one of the key Django developers, James Bennett.

I've followed many of the James work, and it's awesome; but I've a complaint on Django Snippets. It's the inefficient way to find anything there. The only ways that I've found is searching using a paginated list by author, language or tag (sorted alphabetically), where you can spend a huge amount of time if the tag you search starts by Z.

Of course that I can go to Google an search using site:www.djangosnippets.org, and it would work for all indexed snippets, but I think that at least this search box should be added on the site (until anybody writes the generic search engine for Django). It would also help adding result page numbers (with links) at the end of the paginated list.

Changing language on the admin

When working on multilanguage sites, a feature that many times I missed is a direct way to change the language when you're on the admin,

With standard Django the only way that exists (as far as I know), is leaving the admin, going to the website, change the language there, and come back to the admin. Not very fast.

Today, I've created a snippet that creates a drop down menu on the admin bar (just in the main page) to change the language.

Hope it helps.

Saturday, May 10, 2008

DSNP 0.9 released

I know that it was just few days ago that I released another version of DSNP, but because of it, I got a lot of feedback on it, I worked hard, and finally DSNP is stable.

For now, it'll be 0.9 because it works on Django newforms-admin branch, so 1.0 will be reached when newforms-admin will be merged into trunk.

Changeset for this version is next:

  • Generated sqlite file has write permissions for all users by default

  • Static files are served by Django http server on development environment (DEBUG==True)

  • File admin.py created to specify admin options

  • Media path has changed (now, all static files are under "media" directory) due to an issue


I hope you like it.

Friday, May 9, 2008

Unable to define my urls exactly my way (resignation statement)

I've been working with Django for a while, and it helped me to get all my web developments with nice urls. But I also wanted a nice url structure...

What it is nice for me, opposite to Django default settings, is avoiding the media prefix on my media urls, so

http://localhost/media/admin/css/login.css would be http://localhost/admin/css/login.cssTrying to emulate old website structures (used widely in php sites).

That could be nice or not, but for sure it is complicated.

The first step was to setup apache for it, a little bit more complicated than the usual setup, but possible:

<LocationMatch "/((css|js|img|swf|pdf)/|favicon.ico)">
SetHandler None
</LocationMatch>

The main problem comes when using Django development http server (started by "python manage.py runserver"), and the admin. Of course you can do that, but what is not possible, is to define the same name for your admin media path, and for the admin itself.

For example:

http://localhost/admin and http://localhost/admin/css/login.css

The reason is the Django web server, processes all requests starting with the ADMIN_MEDIA_PREFIX setting with the AdminMediaHandler, what implies with that structure that all admin requests (even the ones that aren't static files) are processed by this handler, raising an error when the request isn't for a static file. The error is next.

Permission denied: ${PYTHON_PATH}/django/contrib/admin/media/

So this is my resignation statement to do what I wanted to do initially. Now, options are:

  • Don't use the Django http server (use apache even for development)

  • Keep the same structure but change the admin media directory (for example from "admin" to "admin-media"

  • Use the "media" (or any other name) prefix


NOTE: Current version of my project DSNP is affected by this issue. I'll solve it asap.

Wednesday, May 7, 2008

New DSNP version

DSNP is a simple and customizable Python script, that automatically creates a working Django project.

Project (and application) creation in Django are very openend and flexible, but sometimes is useful getting all the work done for you, specially:

  • If you create many Django projects with the same structure.

  • If you're new to Python, and want to see a "hello world!" application working in less than one minute.

  • If you want to check your Django structure with somebody's else (me).


DSNP does exactly that, automates the process of creating projects and applications in Django. The resulting website is a simple project with a single application, ready for start creating models and templates. It's also customizable, to let everyone set their own preferences in the script, and adapt it to your desired structure.

Want to try it (in five simple steps)?

svn checkout http://dsnp.googlecode.com/svn/trunk/
python dsnp.py myproject
cd myproject
python manage.py runserver
Browse http://localhost:8000/

Easy right?

Tuesday, April 29, 2008

Django L10n

This Sunday, I participated in This Week in Django, and tried to give some ideas on Django localization.

Here I'll post some of the ideas of the interview (and some that I missed), for serving as reference:

How to translate your application (quick guide):

  • Mark every text in your application for translation:

    • In models.py, views.py... convert 'my text in just one language' to _('my text to translate'). Don't forget to import _: from django.utils.translation import ugettext_lazy as _

    • In templates, convert <p>Text in english</p> to <p>{% trans 'Text in many languages' %}</p> (also this can be done with blocktrans tag)



  • Go to your project path and create a directory called locale (also you can do that just for an application)

  • Execute ${PATH_TO_DJANGO}/bin/make-messages -l ${LANGUAGE_CODE} (where language code is en for english, es for spanish...)

  • Edit ${PROJECT_PATH}/locale/${LANGUAGE_CODE}/LC_MESSAGES/django.po and set the msgstr variables with the translation of every msgid

  • Run msgfmt django.po -o django.mo (I just realized after the interview that exists a django script complie-messages.py that does that for all .po files)

  • And then you have your application translated. There are some settings in settings.py that need to be set for making it work (USE_I18N = True, set LANGUAGES and LANGUAGE_CODE, and specify the django.middleware.locale.LocaleMiddleware middleware)

  • Then probably you'll want to have your select input with all available languages (or something like that). For it you'll have to add (r'^i18n/', include('django.conf.urls.i18n')) to your urls.py, and from your html send a POST request to /i18n/setlang with the parameter language set to desired language code

  • For more stuff, and detailed information check: http://www.djangoproject.com/documentation/i18n/


Things that IMHO should be improved in Django for a better L10n expirience:

  • Move localflavors outside trunk (to avoid unnecessary translation costs). Every localflavor should come with necessary translations.

  • Create locale settings (besides translations), to set decimal symbol, date and time format, first day of week... and use it automatically for current locale/language.

    • Check tickets #1061, #3940 and #6783 that gives different approaches to this problem.



  • Create translatable CharFields and TextFields. For now django-multilingual and transdb can be used for it.

  • Adding something to select the language in admin (when more than one is available).

  • Haven't checked it too much, but it'll be good if urls could be translated as well.


Finally I want to thank for letting me participate in TWID to Michael Trier, who is a father, husband, software architect, entrepreneur, a great journalist, and a better person. And also to Malcolm Tredinnick, who recommended me to the show (not sure if I deserved the honour), and for his unpayable help and support on my Django work.

Friday, April 25, 2008

New Accopensys website

Today, I'm proud to announce the new version of the Accopensys website. There is still many work to do, but finally this latest version has been published.

Main changes are next:

  • Website is now running on Django

  • Design has been updated

  • Sections and texts have been rewritten to fit new business strategies

Thursday, March 27, 2008

MySQL and encoding

Today I had another encoding problem in my life...

I had a file with sql inserts, encoded with utf-8. My unix terminal, also encoded with utf-8. I had a database as well, and both database and tables encoded with utf-8.

My problem: when executing my sql file to my database, the data encoding was corrupted. There were just one missing piece not encoded with utf-8, MySQL terminal.

To fix it: mysql -u myuser --default_character_set utf8 mydatabase < myfile

Thursday, March 6, 2008

TransDb: Pretty much easier

Today I've released a new version of TransDb, the Django package that allows storing text at database in more than one language (using the same field).

New version is pretty much easier to use, after fixing many bugs, and avoiding the use of a filter in templates.

Now, migrating your single-language application to a multi-language one is very easy, so almost the only thing you've to do is changing your model fields (no data transformation is required, it is done automatically when you translate texts at admin). A full migration procedure is available at project page.

You can find everything at Google's project page.