maemo.org - Talk - [Announcement] Python2.7 for Maemo5

Page 1 of 3

Next >

Show 40 post(s) from this thread on one page

maemo.org - Talk (https://talk.maemo.org/index.php)

- Development (https://talk.maemo.org/forumdisplay.php?f=13)

- - [Announcement] Python2.7 for Maemo5 (https://talk.maemo.org/showthread.php?t=86900)

AapoRantalainen

2012-09-18 08:15

[Announcement] Python2.7 for Maemo5

Now in extras-devel.

Currently Maemo5 has python2.5. There are some interesting python2.7 applications which can't be ported to N900.

Check python2.7-minimal from extras-devel.

*Packages are fully optified.
*Python2.5 and python2.7 can be simultaneously installed

There are no yet modules for python2.7, but in some cases it is possible to use modules from python2.5.

It is not intended that existing and working applications use python2.7, but if new application needs python2.7 it states that explicitly.

--------
First test: AdvancedCaching (this is existing application and works great with python2.5)

Code:

/opt/agtl-maemo/core.py

-#!/usr/bin/python

+#!/usr/bin/python2.7



+import sys

+sys.path.append('/usr/lib/pymodules/python2.5')

+sys.path.append('/usr/lib/pymodules/python2.5/gtk-2.0')

+sys.path.append('/usr/lib/python2.5/site-packages/')

+sys.path.append('/usr/lib/python2.5/lib-dynload/')

-> works.
---------
Some existing applications fail to start with python2.7, and I'm suspecting mismatch of old python2.5 modules, so maybe modules (or some of them) must be recompiled against new python. I'm asking You to test and report your thoughts. What modules cause errors?

----------------

Source:
https://gitorious.org/python27-maemo5/python27-maemo5

[EDIT]
ctypes works with version 2.7.3~rc2-2.1-maemo6 (on extras-devel)
[/EDIT]

ivgalvez

2012-09-18 09:05

Re: [WIP] Python2.7 for Maemo5

Is it really optified without the trickery used for Python 2.5?. That would be really a definitive improvement over current installation.

Have you tried to build it with Linaro toolchain and Thumb2?. It could give a boost to Python applications (specially startup times). Then, you could use CSSU's Thumb repository to upload this package (if freemangordon agrees).

AapoRantalainen

2012-09-18 10:29

Re: [WIP] Python2.7 for Maemo5

Quote:

Originally Posted by ivgalvez (Post 1268307)

Is it really optified without the trickery used for Python 2.5?

Yes, it "just" installs itself to /opt/python27
("just" = You can count how many modifications and commits it took to achieve this.)

Quote:

Originally Posted by ivgalvez (Post 1268307)

Have you tried to build it with Linaro toolchain and Thumb2?

Not yet. I'm waiting instructions how to participate thumb2. I really respect extras-devel way, i.e. developers send source code and very same builder builds deb-packages.

D@vIcHoJD

2012-09-19 02:48

Re: [WIP] Python2.7 for Maemo5

This is an excelent news, because python 2.5 have any problem with any packages for example wifite_r85, pyrit and others. This required python 2.7. Wait that you compiled python2.7 completely and upload to repositories.

Here are some errors that usually come with some applications using python:(with pyrit):D:D

Code:

/home/user # pyrit

Traceback (most recent call last):

   File "/usr/bin/pyrit", line 4, in <module>

          import pyrit_cli

   File "isr/bin/pyrit_cli.py", line 32, in <module>

          import cpyrit.cpyrit

ImportError: No module named cpyrit.cpyrit

Something similar occurs with other packages
I would like to continue testing the improvements that are implemented with python

Regards.

freemangordon

2012-09-19 17:51

Re: [WIP] Python2.7 for Maemo5

@AapoRantalainen - check your debian/control file (along with others that may be affected) in the package you sent to autobuilder, it seems line endings are windoze style.

freemangordon

2012-09-19 18:02

Re: [WIP] Python2.7 for Maemo5

Quote:

Originally Posted by AapoRantalainen (Post 1268344)

Not yet. I'm waiting instructions how to participate thumb2. I really respect extras-devel way, i.e. developers send source code and very same builder builds deb-packages.

Hmm, what instructions do you need?

StefanL

2012-09-19 19:22

Re: [WIP] Python2.7 for Maemo5

Quote:

Originally Posted by D@vIcHoJD (Post 1268803)

I did not have many issues with running wifite_r85 on python 2.5, more info check here.

T.Paaatrick

2012-09-19 19:27

Re: [WIP] Python2.7 for Maemo5

Thank You.

AapoRantalainen

2012-09-20 04:58

Re: [WIP] Python2.7 for Maemo5

Quote:

Originally Posted by freemangordon (Post 1269194)

@AapoRantalainen - check your debian/control file (along with others that may be affected) in the package you sent to autobuilder, it seems line endings are windoze style.

I send debian/control.in and builder generates debian/control.

For testing I sent only debian-directory without source files, and then I get error log as expected. I will continue investigation.

freemangordon

2012-09-20 05:44

Re: [WIP] Python2.7 for Maemo5

Quote:

Originally Posted by AapoRantalainen (Post 1269403)

I send debian/control.in and builder generates debian/control.

For testing I sent only debian-directory without source files, and then I get error log as expected. I will continue investigation.

there is debian/control in .tar.gz file you have sent to autobuilder. And this is what autobuilder tries to parse before executing debian/rules.

AapoRantalainen

2012-09-20 18:39

Re: [WIP] Python2.7 for Maemo5

Python2.7 hit in extras-devel.
You might want install package named python2.7-minimal (This is package which contains python2.7-binary).

---
When pushing to extras-devel, I only use scp/rcp. When I (accidentally) checked this page: https://garage.maemo.org/extras-assistant/index.php I can see one of my trials on the queue (over six days now). This explains why there wasn't any log (but it doesn't explain why it got stuck on queue)

marmistrz

2012-09-26 15:14

Re: [WIP] Python2.7 for Maemo5

Quote:

Originally Posted by AapoRantalainen (Post 1269754)

Aapo, could you make some way like that: if an env. variable, let's call it USE_PYTHON27, (when calling python) then /usr/bin/python2.7 is called, the other way python2.5 is used.

Is this possible?

taixzo

2012-09-26 16:20

Re: [Announcement] Python2.7 for Maemo5

Is it possible to use pip or easy_install with python2.7 so as to have access to all of the Python packages?

AapoRantalainen

2012-09-27 05:04

Re: [Announcement] Python2.7 for Maemo5

Quote:

Originally Posted by taixzo (Post 1272536)

Is it possible to use pip or easy_install with python2.7 so as to have access to all of the Python packages?

I haven't used them. Seems easy_install is dead. Ubuntu ships easy_install inside package python-setuptools which is actually distribute: http://pypi.python.org/pypi/distribute
----
Pip is http://www.pip-installer.org/en/latest/
Installing it needs distribute or setuptools.
-----

http://cc.oulu.fi/~rantalai/maemo5/python27-setuptools/

So there are python-pkg-resources_0.6.24-1-maemo1_all.deb and python-setuptools_0.6.24-1-maemo1_all.deb (from Ubuntu) made against optified python2.7. Install python2.7 and then these both. Use

Code:

sudo gainroot

/opt/python27/bin/easy_install

-> report what happens and where it downloads things (and will they work). I'm currently running totally out of rootfs and seems it at least needs some temporary space for downloads.

taixzo

2012-09-27 05:32

Re: [Announcement] Python2.7 for Maemo5

Quote:

Originally Posted by AapoRantalainen (Post 1272780)

-> report what happens and where it downloads things (and will they work). I'm currently running totally out of rootfs and seems it at least needs some temporary space for downloads.

Well, it seems I too am running out of rootfs...I'll test this once I've uninstalled a few things.

int_ua

2012-09-27 12:28

Re: [Announcement] Python2.7 for Maemo5

I can't see the package in FAM. Any suggestions?

AapoRantalainen

2012-09-27 15:46

Re: [Announcement] Python2.7 for Maemo5

Quote:

Originally Posted by int_ua (Post 1272916)

I can't see the package in FAM. Any suggestions?

It's section is not "user/".

marmistrz

2012-09-27 19:27

Re: [Announcement] Python2.7 for Maemo5

Quote:

Originally Posted by int_ua (Post 1272916)

I can't see the package in FAM. Any suggestions?

use category "all packages (advanced)"

marmistrz

2012-10-18 15:01

Re: [Announcement] Python2.7 for Maemo5

Can anyone create pyside for python2.7?

MartinK

2012-10-18 16:25

Re: [Announcement] Python2.7 for Maemo5

Quote:

Originally Posted by marmistrz (Post 1282154)

Can anyone create pyside for python2.7?

Good point. :)
+1

Doctor Colossus

2012-10-29 01:50

Re: [Announcement] Python2.7 for Maemo5

PyQT would also be appreciated. This would allow me to finally run Anki 2 -- the new version of the main program I use my N900 for. I tried at length to get qmake working on my device so I could compile it myself, but it always complained:

Code:

/usr/bin/ld: cannot find -lQtCore

marmistrz

2012-10-29 16:09

Re: [Announcement] Python2.7 for Maemo5

Quote:

Originally Posted by Doctor Colossus (Post 1286879)

Code:

/usr/bin/ld: cannot find -lQtCore

I guess you need to use scratchbox
/edit: do you have libqt4-dev installed?

aprilmay

2012-12-10 14:32

Re: [Announcement] Python2.7 for Maemo5

Hi,

First, good to see that there's some effort for porting Python 2.7, thanks a lot to all who are working on it!

I have installed the version 2.7.3~rc2-2.1-maemo4, and facing some problem with the ctypes module, which is required in many applications (eg Twisted).

Even a simple "import ctypes" crashes with "Aborted" message.

I have not tried to debug it, just reporting. Can someone else try and validate this is a genuine problem?

pierrem

2013-05-29 21:02

Re: [Announcement] Python2.7 for Maemo5

Did someone try "import ctypes" in a python 2.7 console ?

it gives this wonderful result:
Aborted

I traced this problem when trying to make weboob (WEB Outside Of Browsers) works on my N900, and it seems that any weboob backend using PIL, and more specifically ctypes, gives the above result.

I don't have the required skills to go futher in my investigation: does someone manage to understand why this could happen, and possibly propose a solution ?

thanks

EDIT: sorry I did not read the aprilmay post above, so obviously this has been reported several month ago

pierrem

2013-06-04 14:03

Re: [Announcement] Python2.7 for Maemo5

I found this page "Cross compiling python for embedded linux" with a patch fixing specifically ctypes for python 2.7.

Maybe, AapoRantalainen could try to make a new release with this fix if it was not already applied in the currently available package, or even use the other patches to compile python 3.x

seanmcken

2013-06-04 15:23

Re: [Announcement] Python2.7 for Maemo5

noob question, plz answer

what is advantage of version 2.7?

pierrem

2013-06-04 15:41

Re: [Announcement] Python2.7 for Maemo5

none if you're only using python 2.5 compliant scripts.
If you want to install the latest version of your preferred python software, odds are high that python 2.5 support has been abandoned in favor of 2.7

AapoRantalainen

2013-06-18 10:20

Re: [Announcement] Python2.7 for Maemo5

I got python2.7 ctypes working with this: http://whocares.de/finally-a-fix-for...ort-of-ctypes/ (I have tried to fix the cause of this without success)

I modified python2.7 from extras-devel and pushed it back to the builder, but seems builder is not in good shape currently and I'm not getting success nor failure (there are several packages built but not moved to the repository which implies some kind of issues).

Here is locally built debs for early adopters: http://www.cs.helsinki.fi/u/rantalai/maemo5/python2.7/

freemangordon

2013-08-15 08:50

Re: [Announcement] Python2.7 for Maemo5

Couldn't find a bugtracker link so posting here

Code:

Core was generated by `/home/fmg/embedlite/mozilla-central/objdir-armel/_virtualenv/bin/python ./confi'.

Program terminated with signal 6, Aborted.

#0  0x42299548 in raise () from /lib/libc.so.6

(gdb) bt

#0  0x42299548 in raise () from /lib/libc.so.6

#1  0x4229ab6c in abort () from /lib/libc.so.6

#2  0x4264edc8 in free (mem=0x423ba6b0)

    at /home/builder4/maemo-fremantle-armel-extras-devel/work/python2.7-2.7.3~rc2-2.1/Modules/_ctypes/libffi/src/dlmalloc.c:4272

#3  0x4264bb10 in selinux_enabled_check (start=0x0, length=4096, prot=3, flags=1111204368, fd=-1, 

    offset=0)

    at /home/builder4/maemo-fremantle-armel-extras-devel/work/python2.7-2.7.3~rc2-2.1/Modules/_ctypes/libffi/src/closures.c:151

#4  dlmmap (start=0x0, length=4096, prot=3, flags=1111204368, fd=-1, offset=0)

    at /home/builder4/maemo-fremantle-armel-extras-devel/work/python2.7-2.7.3~rc2-2.1/Modules/_ctypes/libffi/src/closures.c:456

#5  0x4264c344 in sys_alloc (size=32, code=0x44fb98)

    at /home/builder4/maemo-fremantle-armel-extras-devel/work/python2.7-2.7.3~rc2-2.1/Modules/_ctypes/libffi/src/dlmalloc.c:3458

#6  dlmalloc (size=32, code=0x44fb98)

    at /home/builder4/maemo-fremantle-armel-extras-devel/work/python2.7-2.7.3~rc2-2.1/Modules/_ctypes/libffi/src/dlmalloc.c:4188

#7  ffi_closure_alloc (size=32, code=0x44fb98)

    at /home/builder4/maemo-fremantle-armel-extras-devel/work/python2.7-2.7.3~rc2-2.1/Modules/_ctypes/libffi/src/closures.c:538

#8  0x42643358 in _ctypes_alloc_callback (callable=0x464cf0, converters=0x423bc030, restype=0x54ef78, 

    flags=257)

---Type <return> to continue, or q <return> to quit---

    at /home/builder4/maemo-fremantle-armel-extras-devel/work/python2.7-2.7.3~rc2-2.1/Modules/_ctypes/callbacks.c:426

#9  0x42641190 in PyCFuncPtr_new (type=0x56f748, args=0x471fd0, kwds=0x0)

    at /home/builder4/maemo-fremantle-armel-extras-devel/work/python2.7-2.7.3~rc2-2.1/Modules/_ctypes/_ctypes.c:3482

#10 0x000b3828 in ?? ()

#11 0x000b3828 in ?? ()

Backtrace stopped: previous frame identical to this frame (corrupt stack?)

This started to happen after the latest mozilla upstream merge I did, was fine before that.

Code:

[sbox-FREMANTLE_ARMEL_GCC472: ~/embedlite/mozilla-central/objdir-armel] > dpkg -l | grep python

ii  python2.7                                           2.7.3~rc2-2.1-maemo4                      Interactive high-level object-oriented langu

ii  python2.7-dbg                                       2.7.3~rc2-2.1-maemo4                      Debug Build of the Python Interpreter (versi

ii  python2.7-minimal                                   2.7.3~rc2-2.1-maemo4                      Minimal subset of the Python language (versi

Anything else you need me to provide?

freemangordon

2013-08-15 10:56

Re: [Announcement] Python2.7 for Maemo5

Quote:

Originally Posted by AapoRantalainen (Post 1352867)

Hmm, with this version I see no more aborts, so scratch the above backtrace.

Do you mind to put that in the repos? Increasing the version ofc, as it is the same as the latest in extras-devel :)

AapoRantalainen

2013-08-15 16:02

Re: [Announcement] Python2.7 for Maemo5

Quote:

Originally Posted by freemangordon (Post 1367398)

Do you mind to put that in the repos? Increasing the version ofc, as it is the same as the latest in extras-devel :)

I have tried to put this fixed version to the extras-devel, but it silently rejects it. Can you fetch some logs for me?

I should use bigger version number for fixed version (and not same than non-fixed version).

freemangordon

2013-08-15 17:12

Re: [Announcement] Python2.7 for Maemo5

Quote:

Originally Posted by AapoRantalainen (Post 1367446)

Hmm, those logs are available to everyone: https://garage.maemo.org/builder/fremantle/

However, could you reupload python2.7_2.7.3~rc2-2.1-maemo5 (I guess this is the fixed version) to the builder again, I have the feeling your attempt coincide with some maintaining or reboot. Try first without increasing the version.

EDIT:
If it fails again, I'll look at it more closely

EDIT2:
And I'll appreciate if I receive reports for anything suspicious related to the autobuilder/package interface ;)

EDIT3:
don;t reupload to builder, there are some problems, will inform you when the builder is fixed

AapoRantalainen

2013-08-16 07:28

Re: [Announcement] Python2.7 for Maemo5

Now I can see builder logs (you did something?)

armel build ok: 04-Aug-2013 21:17
i386 failed: 15-Aug-2013 18:07

(Reading from logs) During tests on arm qemu seg faults and then skips tests and success with packages.
On i386 it runs tests and fails with "Name or service not known" during test_docxmlrpc (SimpleXMLRPCServer.py+SocketServer.py) which looks something that builder might block for reason.

And actually it is not tests, but run_profile_task on

Code:

profile-opt:

        @echo "Building with support for profile generation:"

        $(MAKE) clean

        $(MAKE) build_all_generate_profile

        @echo "Running benchmark to generate profile data:"

        $(MAKE) profile-removal

        $(MAKE) run_profile_task

        @echo "Rebuilding with profile guided optimizations:"

        $(MAKE) clean

        $(MAKE) build_all_use_profile

And debian/rules is explicitly asking profiled build (so it can easily disabled).

freemangordon

2013-08-16 08:56

Re: [Announcement] Python2.7 for Maemo5

Quote:

Originally Posted by AapoRantalainen (Post 1367594)

Code:

profile-opt:

        @echo "Building with support for profile generation:"

        $(MAKE) clean

        $(MAKE) build_all_generate_profile

        @echo "Running benchmark to generate profile data:"

        $(MAKE) profile-removal

        $(MAKE) run_profile_task

        @echo "Rebuilding with profile guided optimizations:"

        $(MAKE) clean

        $(MAKE) build_all_use_profile

And debian/rules is explicitly asking profiled build (so it can easily disabled).

Yep, I did, there was a make -j4 ... stuck since 4th of August. We suspect OOM, now there is more memory added to builder VM. I killed that stuck make, so it continued, albeit failed.

Could you upload python2.7 again, to see if it will work this time.

freemangordon

2013-08-16 10:31

Re: [Announcement] Python2.7 for Maemo5

@aapo - Definitely it is not autobuilder to blame, for some reason a python process got stuck. I killed it by hand, so the build will succeed, but I think you should check the build rules and disable whatever is not really needed. Like profiled build, tests, etc.

EDIT:

Code:

Exception in thread Thread-268:

Traceback (most recent call last):

  File "/home/builder1/maemo-fremantle-i386-extras-devel/work/python2.7-2.7.3~rc2-2.1/Lib/threading.py", line 551, in __bootstrap_inner

    self.run()

  File "/home/builder1/maemo-fremantle-i386-extras-devel/work/python2.7-2.7.3~rc2-2.1/Lib/logging/config.py", line 883, in run

    ready=self.ready)

  File "/home/builder1/maemo-fremantle-i386-extras-devel/work/python2.7-2.7.3~rc2-2.1/Lib/logging/config.py", line 851, in __init__

    ThreadingTCPServer.__init__(self, (host, port), handler)

  File "/home/builder1/maemo-fremantle-i386-extras-devel/work/python2.7-2.7.3~rc2-2.1/Lib/SocketServer.py", line 408, in __init__

    self.server_bind()

  File "/home/builder1/maemo-fremantle-i386-extras-devel/work/python2.7-2.7.3~rc2-2.1/Lib/SocketServer.py", line 419, in server_bind

    self.socket.bind(self.server_address)

  File "/home/builder1/maemo-fremantle-i386-extras-devel/work/python2.7-2.7.3~rc2-2.1/Lib/socket.py", line 224, in meth

    return getattr(self._sock,name)(*args)

gaierror: [Errno -2] Name or service not known



make[2]: *** [run_profile_task] Terminated

Terminated is because of me killing the stuck python

AapoRantalainen

2013-08-16 11:05

Re: [Announcement] Python2.7 for Maemo5

Built success. Waiting package coming to the repository for testing.

Quote:

Originally Posted by freemangordon (Post 1367624)

@aapo - Definitely it is not autobuilder to blame, for some reason a python process got stuck.

How about this, happens on builder for ARM, but not my local SB.

Code:

test_exceptions

qemu: uncaught target signal 11 (Segmentation fault) - core dumped

make[2]: [run_profile_task] Error 126 (ignored)

Quote:

Originally Posted by freemangordon (Post 1367624)

you should check the build rules and disable whatever is not really needed. Like profiled build, tests, etc.

I have tried to do that, but this time that profiled build was slipped out of my radar.

freemangordon

2013-08-16 11:14

Re: [Announcement] Python2.7 for Maemo5

Quote:

Originally Posted by AapoRantalainen (Post 1367634)

Built success. Waiting package coming to the repository for testing.

How about this, happens on builder for ARM, but not my local SB.

Code:

test_exceptions

qemu: uncaught target signal 11 (Segmentation fault) - core dumped

make[2]: [run_profile_task] Error 126 (ignored)

I have tried to do that, but this time that profiled build was slipped out of my radar.

qemu, only devils know how it works, not the God even :).

I guess your SB setup differs from autobuilder.

Code:

ii  scratchbox-core:i386                               1.0.18                       Scratchbox base system

ii  scratchbox-devkit-apt-https:i386                   1.0.11                       APT HTTPS devkit for Scratchbox

ii  scratchbox-devkit-debian-squeeze:i386              1.0.4                        Debian Squeeze devkit for Scratchbox

ii  scratchbox-devkit-doctools:i386                    1.0.14                       Doctools for Scratchbox

ii  scratchbox-devkit-git:i386                         1.0.2                        Git for Scratchbox

ii  scratchbox-devkit-perl:i386                        1.0.5                        Perl modules for Scratchbox

ii  scratchbox-devkit-qemu:i386                        0.12.50-0sb3                 Qemu scratchbox devkit

ii  scratchbox-libs:i386                               1.0.18                       Scratchbox libraries

ii  scratchbox-toolchain-cs2005q3.2-glibc2.5-arm:i386  1.0.7.2                      cs2005q3.2-glibc2.5-arm compiler for Scratchbox

ii  scratchbox-toolchain-cs2005q3.2-glibc2.5-i386:i386 1.0.7                        cs2005q3.2-glibc2.5-i386 compiler for Scratchbox

ii  scratchbox-toolchain-cs2007q3-glibc2.5-arm7:i386   1.0.16-2                     cs2007q3-glibc2.5-arm7 compiler for Scratchbox

ii  scratchbox-toolchain-cs2007q3-glibc2.5-i486:i386   1.0.16-1                     cs2007q3-glibc2.5-i486 compiler for Scratchbox

ii  scratchbox-toolchain-host-gcc:i386                 1.0.18                       Scratchbox host-gcc toolchain

and fremantle scratchbox in autobuilder is using debian-squeeze, not debian-etch

AapoRantalainen

2013-08-16 19:06

Re: [Announcement] Python2.7 for Maemo5

Builder got -maemo5 built (arm+i386) , but there were also stucked -maemo6, so it didn't push it to the repository.

I bumped version number and reuploaded, but seems i386 version is again exhausted, armel deps are already in results: https://garage.maemo.org/builder/fre...c2-2.1-maemo6/

freemangordon

2013-08-18 19:45

Re: [Announcement] Python2.7 for Maemo5

Quote:

Originally Posted by AapoRantalainen (Post 1367740)

For some reason it hangs on "./python ../Lib/test/regrtest.py -x test_bsddb3 test_compiler test_cprofile test_dbm_dumb test_dbm_ndbm...", I guess next time you should disable regression tests in the build scripts.

There seems to be another problem with python2.7, similar to the KP problem - newest version does not appear in the package interface as the latest version. I'll look at that problem as soon as I have some spare time.

AapoRantalainen

2013-08-19 06:26

Re: [Announcement] Python2.7 for Maemo5

Quote:

Originally Posted by freemangordon (Post 1368079)

I guess next time you should disable regression tests in the build scripts

I think I now got how to disable it correctly (for next time).

Quote:

Originally Posted by freemangordon (Post 1368079)

There seems to be another problem with python2.7, similar to the KP problem - newest version does not appear in the package interface as the latest version. I'll look at that problem as soon as I have some spare time.

Yes, this is weird. But now working package is in extras-devel. And package page is: http://maemo.org/packages/package_in...c2-2.1-maemo6/
Is there risk that builder will remove it as a old version? Should I save it to the testing?

All times are GMT. The time now is 05:13.

Page 1 of 3

Next >

Show 40 post(s) from this thread on one page

vBulletin® Version 3.8.8