Pandas build failing on Django AWS Elastic Beanstalk deployment

Question

I haven't had issues previously deploying updates to my application; however, today I tried to deploy a branch of my code to a new environment and I keep receiving this error:

building 'pandas.algos' extension
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -Ipandas/src/klib -Ipandas/src -I/opt/python/run/venv/lib/python2.6/site-packages/numpy/core/include -I/usr/include/python2.6 -c pandas/algos.c -o build/temp.linux-x86_64-2.6/pandas/algos.o

In file included from /opt/python/run/venv/lib/python2.6/site-packages/numpy/core/include/numpy/ndarraytypes.h:1728,
from /opt/python/run/venv/lib/python2.6/site-packages/numpy/core/include/numpy/ndarrayobject.h:17,
from /opt/python/run/venv/lib/python2.6/site-packages/numpy/core/include/numpy/arrayobject.h:15,
from pandas/algos.c:314: /opt/python/run/venv/lib/python2.6/site-packages/numpy/core/include/numpy/npy_deprecated_api.h:11:2: warning: #warning "Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION"

pandas/algos.c: In function __pyx_pf_6pandas_5algos_450group_ohlc_float64:
.... (other warnings)
pandas/algos.c:118189: warning: __pyx_v_vclose may be used uninitialized in this function

{standard input}: Assembler messages:
{standard input}:2324793: Warning: end of file in string; '"' inserted
....
gcc: Internal error: Killed (program cc1)
error: command 'gcc' failed with exit status 1

I have also seen a different narrative for the error:

{standard input}:78968: Warning: end of file not at end of a line; newline inserted
{standard input}:79677: Error: open CFI at the end of file; missing .cfi_endproc directive

Pandas 0.12.0 (and NumPy 1.7.1 etc.) are all included in my requirements.txt. Has something changed or am I doing something wrong here? Everything is working fine in my local environment.

greenafrican · Accepted Answer · 2013-09-20 20:47:57Z

4

I didn't find many answers to this issue but I did find a post related to similar 'gcc' builds and this EOF error. It suggested increasing memory so I changed the AWS EC2 spec and managed to build and deploy. I was trying to build a t1.micro and found that I need to up that to an m1.small.

I'd like to know how I can best determine the required resources for my environment build. I will start another thread posing that as a question.

answered Sep 20, 2013 at 20:47

greenafrican

2,5565 gold badges29 silver badges39 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Alex P. Miller Over a year ago

Thanks, this solution worked great for me. I might suggest changing your "django" tag to just a "python" tag, since this isn't really related to Django. I'm actually running Flask on Amazon EC2, but this solution is exactly what I needed. Also, I was able to upgrade to a higher tier, run the installation, and downgrade back to t1.small. I was able to run "import pandas" with no problem.

greenafrican Over a year ago

@alexpmil yeah I did the exact same thing. Upgraded, installed and then downgraded. Anyway, i'm now running on the m3.large so it isn't an issue.

SiggyF · Accepted Answer · 2014-07-13 21:46:04Z

2

Another approach is to enable some swap memory. This results in a slow but successful compilation.

# create 2 GB swap
dd if=/dev/zero of=/var/swap.1 bs=1M count=2048
# format 
mkswap /var/swap.1
# turn on swap memory
swapon /var/swap.1

You can add the following line to /etc/fstab to enable swap after a reboot

/var/swap.1 swap swap defaults 0 0 to /etc/fstab

I'm not sure what the logical place is for the swap file. Normally you'd use a separate partition. I've seen others use /swapfile.

answered Jul 13, 2014 at 21:46

SiggyF

23.3k8 gold badges46 silver badges57 bronze badges

Comments

axplusb · Accepted Answer · 2015-10-14 21:34:50Z

2

I had to do both:

upgrading from a t2.micro to a t2.small EC2 instance
put
yum: gcc-c++: [] python27-devel: []
in my .ebextenstions/whatever.config

And that seems to do the trick!

answered Oct 14, 2015 at 21:34

axplusb

4594 silver badges9 bronze badges

Collectives™ on Stack Overflow

Pandas build failing on Django AWS Elastic Beanstalk deployment

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

2 Comments

Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related