IT46 Blog

April 20, 2006 16:17

Compiling Mozilla in OpenOffice.org 2.0.3 Win32

I am compiling OpenOffice.org OOB680m5 (2.0.2) under Win32 with cygwin and tcsh. As part of the build process, Mozilla 1.7.5 autoconf breaks. I got the following error:

cd ./wntmsci10.pro/misc/build/mozilla/ && sh -c "./configure  --disable-tests --<snip>
creating cache ./config.cache
checking host system type... i586-pc-msvc
checking target system type... i586-pc-msvc
checking build system type... i586-pc-msvc
checking for gcc... cl.exe
checking whether the C compiler (cl.exe  ) works... yes
checking whether the C compiler (cl.exe  ) is a cross-compiler... no
checking whether we are using GNU C... no
checking whether cl.exe accepts -g... no
checking for c++... cl.exe
checking whether the C++ compiler (cl.exe  ) works... yes
checking whether the C++ compiler (cl.exe  ) is a cross-compiler... no
checking whether we are using GNU C++... no
checking whether cl.exe accepts -g... no
checking for ranlib... ranlib
checking for ml... /cygdrive/i/VS2003/Vc7/bin/ml
checking for ar... ar
checking for ld... link
checking for strip... strip
checking for windres... windres
checking for gawk... gawk
checking for midl... midl
configure: error: This version of the MSVC compiler,  , is unsupported.

Having a look into the configure, i found the following lines of dirty code:

CC_VERSION=`"${CC}" -v 2>&1 | grep Version | sed -e 's|.* Version ||' -e 's| .*||'`

CXX_VERSION=`"${CXX}" -v 2>&1 | grep Version | sed -e 's|.* Version ||' -e 's| .*||'`

_LD_FULL_VERSION=`"${LD}" -v 2>&1 | grep Version | sed -e 's|.* Version ||' -e 's| .*||'`

_MIDL_FULL_VERSION=`"${MIDL}" -v 2>&1 | grep Version | sed -e 's|.* Version ||' -e 's| .*||'`

My environment looks as follows:

  • CC_VERSION cl.exe Compilador de optimización versión 13.10.3077 para 80x86 de Microsoft (R) C/C++ de 32 bits (Spanish)
  • CXX_VERSION cl.exe Compilador de optimización versión 13.10.3077 para 80x86 de Microsoft (R) C/C++ de 32 bits (Spanish)
  • _LD_FULL_VERSION link.exe Microsoft (R) Incremental Linker Version 7.10.3077 (English)
  • _MIDL_FULL_VERSION Microsoft (R) 32b/64b MIDL Compiler Version 6.00.0361 (English)
  • .NET Microsoft (R) Visual C# .NET-kompilator version 7.10.6001.4 (Swedish)

The binaries are:

$ which link
/cygdrive/i/VS2003/Vc7/bin/link
$ which cl
/cygdrive/i/VS2003/Vc7/bin/cl
$ which midl
/cygdrive/i/VS2003/Common7/Tools/Bin/midl
$ which csc
/cygdrive/i/MFNET/csc

The problem takes place in the way that autoconf in Mozilla finds the version, as it is expecting an ENGLISH version of the Visual Studio 2003. The fast-fix looks as follows:

CC_VERSION=`"${CC}" -v 2>&1 | grep versión | sed -e 's|.* versión ||' -e 's| .*||'`

More issues:

  • localize_sl.exe crashes under Win32 http://l10n.openoffice.org/servlets/BrowseList?list=dev&by=thread&from=1193032
  • #49856: Remove $PSDK_HOME/lib/libcp.lib

Read more »

April 17, 2006 09:46

Adding a new collation algorithm

The i18npool library was containing large amount of data for Asian languages which made memory foot print big for western languages. New collation algorithms were added to the file

i18npool/inc/collator_unicode.hxx


COLLATOR( xx_charset )

Currently the data resides in a external library and loaded on demand. Patches to add new collators should be applied to the file:

http://l10n.openoffice.org/source/browse/l10n/i18npool/source/collator/data/collator_data.map

For more info, see: http://www.openoffice.org/issues/show_bug.cgi?id=60331

OBS! http://www.khmeros.info/tools/Collation_in_ooo_2.0.html Documentation needs to be upgraded to reflect this

Read more »

April 14, 2006 20:40

Remote Procedure Call method to Build OOo

I am interested in connecting a Pottle translation server with the OOo build server. Ideally, translation teams should have the chance to rebuild their localized software frequently, so as to evaluate their work continuosly. Currently, most of the localization teams of OpenOffice.org do not have access to a running version of their work until they complete big parts of translation. Providing an easy method to rebuild the software can improve the Quality of the overall translation work.

The main idea behind this, it is to allow a Translation Editor as Pootle the possibility of requesting a test-build of OOo for certain language. I have been investigating the different approaches to RPCs or remote procedure calls. After checking XML-RPC, CORBA, DCOM and SOAP. I found that XML-RPC was the easier approach to the problem.

XML-RPC is a portable way to make remote procedure calls over HTTP. There are several implementations of XML-RPC in Perl, Java, Python, C, C++ and PHP.

I decided to use python in the client side as Pootle is written in python and eventually the client code can be integrated smoothly. The standard Python library includesxmlrpclib; so no extra libraries are needed.In the server side i am using PHP4. XML-RPC support in PHP is not enabled by default and requires the use of --with-xmlrpc[=DIR]; configuration option when compiling PHP to enable XML-RPC support. This extension is bundled into PHP as of 4.1.0.

The are other alternative to the built-in XML-RPC function (See Sourceforge XMLRPC project). One good alternative is the use of Keith Devens PHP XMLRPC Libraries. Based on the later, I wrote a proof of concept. Here it is a short description of what the code is doing right now:

XML-RPC for OOo Build Server

I have defined three XML-RPC methodNames described as following:

  • ooobuild: This method requests the action: build or update of certain version of OOo for certain language and provides a mechanism to notify to the user when is completed. It uses the following parameters:
    • software: OOo 2.0.3
    • action=update,build: update
    • ISO language: sw-TZ
    • resource URI: Place where to find the GSI file, transport can be ftp:// or http://, compression can be bzip2 or gz
    • notification URI: e-mail
    The response, gives acknowledment info message(s) and ID of the requested task idtask.
  • ooostatus: This method requests information about the status of a task with certain ID. It accepts one parameter: idtask and returns the status of the task
  • ooocancel: This method requests the cancelation of a ooobuild. It accepts one parameter: idtask

Code examples

All the code is available here. The code if the XML-RPC ooobuild client in Python looks as follows:

import xmlrpclib

server_url = 'http://127.0.0.1/rpc/ooo_xmlrpc/server.php';
server = xmlrpclib.Server(server_url);

requestdata=('OOo 2.0.3','update','sw-TZ','http://www.it46.se/downloads/rpc/GSI_sw-TZ.sdf.bz2','aep@it46.se')

#Calls the XML-RPC Server to the method ooobuild
result = server.ooobuild(requestdata)

#print result
#It returns an acknowledge of the parameters received and result[5] returns the ID of the task
#ID task can be the UNIX time so we can use it as a timestamp
print "--------------------------------------------------"
print "Product:", result[0]
print "Action:", result[1]
print "Language ISO code:", result[2]
print "Resource:", result[3]
print "Notification:", result[4]
print "ID Task:", result[5]
print "--------------------------------------------------"

Read more »

April 7, 2006 19:12

Building OpenOffice.org, too fast-too furious

Building OpenOffice.org, too fast-too furious

% of build time for OpenOffice.org 2.0.2

It has been more than a year now that i have been building OpenOffice.org, after all this time I have even considered printing my own T-shirts with the text: I build OpenOffice.org, do you?. After spending more than 500 hours building the software (yes, it is sad becouse it is like 20 full-days), i always wondered: where all this time goes?. I finally got the time to figure it out.

Read more »

April 3, 2006 14:22

Fixes, Snygghack, Ñapas (FSÑs) for OpenOffice.org build

Fixes, Snygghack, Ñapas (FSÑs) for OpenOffice.org build

Star-topology in Ethernet

I am posting all the patches i am using in the build system here.

build-ooo

-dt: enables default timestamp when extracting GSI files
localize -e -l sw-TZ -f GSI-sw_TZ_20020202.sdf -dt

OOo 2.0.2

Mozilla

Read more »

April 1, 2006 22:50

OpenOffice.org 2.0.2 OOB680 m5 Swahili

OpenOffice.org 2.0.2 OOB680 m5 Swahili

Swahili Translation of OpenOffice.org in Pootle

I have migrated the existing translations of Jambo OpenOffice.org 1.1.3 to the latest stable version of OOB680m5 following the steps described here. Using the translate toolkit tool pocount as follows:

pocount  `find . -name "*.po" `

got the following statistics:

OpenOffice.org 2.0.2 Swahili (without helpcontent2)

TOTAL:
type           strings words (source) words (translation)
translated:   14191      43766           42876
fuzzy:         2402       4031             n/a
untranslated:  4464      24186             n/a
Total:        21057      71983           42876
File count:     177

OpenOffice.org 2.0.2 Swahili (with helpcontent2)

TOTAL:
type           strings words (source) words (translation)
translated:   17406      49220           48777
fuzzy:         3346       5389             n/a
untranslated: 37122     440426             n/a
Total:        57874     495035           48777

File count:     228

Read more »

March 29, 2006 18:20

Installing PO-based Online Translation and Localization Engine (Pootle)

Installing PO-based Online Translation and Localization Engine (Pootle)

Pootle and the Flumps

This is the notes i took while installing Pootle in the OpenOffice.org build server that i am working with. To get Pootle running you need the following pieces of software: jToolkit, translate toolkit, Pootle, PyLucene (optional) and psyco (optional)

Downloading the software

  • jToolkit (0.7.6): Python web application framework built on modpython and Apache, http://translate.sourceforge.net/snapshots/ or http://jtoolkit.sourceforge.net/
  • Pootle (0.7a1): Online Translation and Localization Engine, cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/translate co -P src
  • Translate Toolkit (0.9a2): Conversion tools for PO files, cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/translate co -P src
  • PyLucene (1.9.1): Lucene's text indexing and searching capabilities from Python, http://downloads.osafoundation.org/PyLucene/src/
  • Psyco (1.5), A just-in-time (JIT) compiler approach for Python, svn co http://codespeak.net/svn/psyco/dist/ psyco-dist

Installing

  • jToolkit (0.7.6)
    # wget http://translate.sourceforge.net/snapshots/jToolkit-0.7.6/jToolkit-0.7.6.tar.gz
    # tar zxvf jToolkit-0.7.6.tar.gz
    # cd jToolkit-0.7.6
    # ./jToolkitSetup.py install
    
  • Pootle (0.7a1)
    # cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/translate co -P src
    # cd src; ./pootlesetup.py install 
    
  • Translate Toolkit (0.9a2)
    # cd src; ./setup install
    
  • PyLucene (1.9.1) (depends on SWIG)
    # wget http://mesh.dl.sourceforge.net/sourceforge/swig/swig-1.3.24.tar.gz
    # tar zxvf swig-1.3.24.tar.gz
    # cd SWIG-1.3.24
    # ./configure
    # make
    # make install
    # wget http://downloads.osafoundation.org/PyLucene/src/PyLucene-src-1.9.1.tar.gz
    # tar zxvf PyLucene-src-1.9.1.tar.gz
    # cd PyLucene-src-1.9.1
    

    I did not manage to get PyLucene to run (segfaults) using gcc/gcj 4.1, so i did have to install a older version of the gcc compiler. I compiled the compiler from the source as follows:

    # wget ftp://mirrors.rcn.net/pub/sourceware/gcc/releases/gcc-3.4.4/gcc-3.4.4.tar.bz2
    # mkdir gcc-3.4.4
    # cd gcc-3.4.4
    # bunzip2 -c ../gcc-3.4.4.tar.bz2 | tar -xvf -
    # mkdir build
    # cd build
    # ../gcc-3.4.4/configure --enable-threads=posix --prefix=/usr/local/gcc-3.4.4 --enable-languages=c,c++,java
    # make bootstrap
    # sudo make install
    

    Once i got gcc 3.4.4 installed, i edit the Makefile of PyLucene as follows:

    # Linux
    PREFIX=/usr/
    PREFIX_PYTHON=$(PREFIX)
    SWIG=/usr/local/bin/swig
    GCJ_HOME=/usr/local/gcc-3.4.4/
    GCJ_VER=3
    ANT=ant
    #DB=$(PYLUCENE)/db-$(DB_VER)
    #DB=/usr/lib/
    #PREFIX_DB=/usr/lib
    
  • Psyco (1.5)
    # wget -r http://codespeak.net/svn/psyco/dist/ -I /svn/psyco/dist 
    # cd codespeak.net/svn/psyco/dist
    # ./setup.py install
    

Read more »

March 27, 2006 19:54

Mozilla breaks with glibc 2.4

Mozilla breaks with glibc 2.4

Dejavu Netscape

In order to enforce portability by glic 2.4pre, the JS_BP and JB_BP are undeclared. The internals of the jmp_buf are platform dependent and programs that use those internals are not ANSI conformant. What does this mean for common mortals? Well, it means that programs that were working in Linux that use the jmp_buf stop working... things breaks!.

In my SuSE 10.1 i am using glibc-2.4 and parts of the Stack walking code in Mozilla (xpcom base) needs some patching work. So, those who build old versions of Mozilla during the OpenOffice.org build need apply the dumpstack patch

While browsing for a picture for this post, i found an interesting project: Dejavu. Check it out! Reference links [1]- [2]- [3]- [4]

Read more »

March 27, 2006 17:28

Building OpenOffice.org in Suse 10.1 Beta8

Building OpenOffice.org in Suse 10.1 Beta8

Multi-lingual Build en,fr,es,sw-TZ

As the spring has not arrived to Sweden and we still live in the -10 C, +1 C range, i have spent this long and dark weekend building OpenOffice.org m160 x86. I have installed the latest release of SuSE 10.1 Beta8 in the Sun X4100 server. Installing SuSE in the X4100 was a smooth process, the stock kernel supports the mtpsas driver (applause!). In fact, it took more time to download the CDs that to get through the installation process (whistles!).

Unfortunatelly, this release lacks many of the development packages needed to build OpenOffice.org. It was not a bad exercise to have to recompile most of the auxiliary packages from sources. Well, at least if you do it once in a life time, so I have filed an Issue #160939 so as to get those RPMs in the distro.

Downloading packages in SuSE using Yast is a real pain. It reminds me those dark times of using dselect. Compiling four languages took around four hours without ccache. I should make some benchmarking for those geeks googling around.

Read more »

March 27, 2006 13:35

Linux Kernel 2.6.16 git includes MPT Fusion

Since the release of the kernel v2.6.14-rc3, MPT fusion SAS support is included in the linux kernel. Kernel version 2.6.16 supports SAS hotswap. Distributions shipping with this kernel should run in the Sun X4100. SuSE 10.1 Beta 8 has been released with Kernel 2.6.16

ooo:/ # uname -a
Linux ooo 2.6.16-rc6-git1-4-smp #1 SMP Tue Mar 14 18:04:33 UTC 2006 i686 athlon i386 GNU/Linux

I have been using the machine for one week now and it seems very stable. No oops... Kernel programmers are very special people: no comments.

	if (panic_on_oops) {
		printk(KERN_EMERG "Fatal exception: panic in 5 seconds\n");
		set_current_state(TASK_UNINTERRUPTIBLE);
		schedule_timeout(5 * HZ);
		panic("Fatal exception");
	}

Read more »

Image Resolution: Small | Medium | Large