Opened 7 years ago

Closed 6 years ago

#54702 closed defect (fixed)

Python 36 and Python 35 crash reading files larger than 2 Gig

Reported by: jhelm (John Helm) Owned by: jmroot (Joshua Root)
Priority: Normal Milestone:
Component: ports Version: 2.4.1
Keywords: Cc:
Port: python35 python36

Description

On OS X 10.11.6 (el Capitan) the following program crashes Python 3.6.2 and 3.5.4 with error "OSError: [Errno 22] Invalid argument" when trying to open files larger than 2.0 Gig; smaller files work fine. Python 2.7.13 does not exhibit the problem. A similar bug was logged at bugs.python.org as Issue 24658, "open().write() fails on 2 GB+ data (OS X)" (https://bugs.python.org/issue24658) and remains open. (Note be sure my install was okay; I uninstalled, reinstalled, and ran port -nR upgrade --force python36)

import    sys

def main():
    lines   = sys.stdin.read()
    cnt     = 0
    for line in lines: cnt += 1
    print("lines",cnt)

if __name__ == "__main__":
    main()


Change History (8)

comment:1 Changed 7 years ago by mf2k (Frank Schima)

Cc: jyrkiwahlstedt removed
Owner: set to jyrkiwahlstedt
Port: python36, python35python35 python36
Status: newassigned

comment:2 Changed 7 years ago by jhelm (John Helm)

Update: I upgraded to xcode 8.2.1 (8C1002) and verified the compiler version to be

jlh-MacBook-Pro:fbaNG01 jlh$ cc --version
Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin15.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

I uninstalled and reinstalled python36, ran 'port diagnose', and then ran 'sudo port -nR upgrade --force python36' to force a rebuild. I still get the error.

Could my path or environment be wrong? Here's echo $PATH

/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/MacGPG2/bin:/Applications/Postgres.app/Contents/Versions/latest/bin

I won't spam you with the environment, but I can see anything egregious.

Finally I've installed python34, python33, and python31; they all exhibit the same problem.

Thanks.

comment:3 Changed 7 years ago by mf2k (Frank Schima)

So this is an upstream bug? I don't see how we can do anything about that.

comment:4 Changed 7 years ago by jhelm (John Helm)

If by upstream bug, you mean the problem is in the distro from python.org, then I think you are correct. I built a OS X vm and installed python from python.org; the bug is still there! I never expected something like this would go undetected... Let's close this bug and I'll file with python.org.

BTW - here is a slowish workaround by reading one line at a time:

    f = sys.stdin
    cnt = 0
    for line in f:
        cnt += 1
    print("lines",cnt)

Thanks for everything.

comment:5 Changed 7 years ago by ryandesign (Ryan Carsten Schmidt)

Do please mention here the URL of the upstream bug report when you file it.

comment:6 Changed 7 years ago by jmroot (Joshua Root)

Owner: changed from jyrkiwahlstedt to jmroot

comment:7 Changed 6 years ago by jmroot (Joshua Root)

Is this still a problem with the current version? Did you file that upstream bug report?

comment:8 Changed 6 years ago by jmroot (Joshua Root)

Resolution: fixed
Status: assignedclosed

In c171978db4a892b60ad65d1d799fae6827962f72/macports-ports (master):

python36: update to 3.6.8

Fixes: #54702

Note: See TracTickets for help on using tickets.