931 questions
0
votes
1
answer
62
views
Broadcasting arrays in Eigen along rows and columns to create a rectangular array
Is there a way to simultaneously broadcast rows and columns?
The following MWE does not work:
#include <iostream>
#include <Eigen/Dense>
int main() {
std::cout << "Hello, ...
- 119
1
vote
1
answer
91
views
Eigen::Array colwise/rowwise broadcasting is asymmetric?
The following code multiplies the i-th column of A by the i-th element of V.
#include <Eigen/Core>
Eigen::Array<double, 8, 2>
f(const Eigen::Array<double, 8, 2>& A, const Eigen::...
- 1,878
1
vote
1
answer
82
views
How to broadcast operation to Numpy array of objects?
Say I have a Numpy array of 500 lists with random sizes ranging from 0 to 9:
import numpy as np
a = np.array([[i for i in range(np.random.randint(10))] for _ in range(500)], dtype=object)
Now I want ...
- 2,965
2
votes
1
answer
76
views
Alternative to looping over one numpy axis
I have two numpy arrays a and b such that a.shape[:-1] and b.shape are broadcastable. With this constraint only, I want to calculate an array c according to the following:
c = numpy.empty(numpy....
- 177
0
votes
2
answers
100
views
Shortest way to broadcast 1d array to specific dimension in NumPy
I often find myself to broadcast 1d arrays into a specific dimension dim_a given a total number of dimension dim_total. What I mean is the following:
import numpy as np
a = np.arange(10)
dim_a = 2
...
- 1,941
2
votes
1
answer
76
views
Boolean Indexing in Numpy Arrays
I was learning boolean indexing in numpy and came across this. How is the indexing below not producing a Index Error as for axis 0 as there are only two blocks?
x = np.arange(30).reshape(2, 3, 5)
x
...
0
votes
0
answers
54
views
Efficient way to operate over a list of Numpy arrays of different sizes
I am writing a function to bin points based on their angle in a radial coordinate system. I would like to have the option to perform some nonlinear downsampling of the points in each bin (computing ...
1
vote
2
answers
43
views
Form element-wise list from scalar and matrix
I have a zero-dimensional numpy scalar s and a two-dimensional numpy matrix m. I want to form a matrix of vectors in which all the elements of m are paired with s as in the following example:
import ...
- 539
-1
votes
1
answer
80
views
Can't vectorize this function - works with constants but returns ValueError operands could not be broadcast together
I wrote a python function I would expect to allow vectorization, using np.where and np.maximum. However, when attempting to call that function by passing dataframe columns, I get the error "...
- 13
0
votes
1
answer
55
views
Can array broadcasting treat larger dimensions as smaller numbers?
I am trying to avoid a for loop by using array broadcasting in order to save time.
My setup is as follows: I have a column array of length n where each element is a mxm array (which I'll call A)
and ...
- 181
0
votes
1
answer
77
views
In NumPy arrays, is there a syntax to set a value in the last dimension based on the first dimension?
I've Googled and asked ChatGPT and looked through NumPy docs and can't find any way to do this, so thought I'd ask here.
Suppose I have a 4-dimensional array -- in this case, of shape (3, 2, 2, 2):
a =...
- 1,375
1
vote
2
answers
102
views
Most 'Pythonic' way of forcing array-like behavior on non-array inputs?
I'm working on a personal project for computing and manipulating integer distributions, for application in systems like DnD and pathfinder.
I'm looking to make functions that can query the probability ...
- 53
3
votes
2
answers
73
views
Expand numpy array to be able to broadcast with second array of variable depth
I have a function which can take an np.ndarray of shape (3,) or (3, N), or (3, N, M), etc.. I want to add to the input array an array of shape (3,). At the moment, I have to manually check the shape ...
- 466
1
vote
1
answer
69
views
Will the NumPy broadcast array ever be created during a binary operation?
I have two numpy.ndarray instances with different shapes. If I add these two arrays, broadcasting will occur between them:
import numpy as np
x = np.array([1, 2, 3])
y = np.array([[2, 3, 5],
...
- 178
0
votes
1
answer
100
views
How can I vectorize this linalg.lstq() operation?
I am trying to implement a multi-frequency phase unwrapping algorithm using Python3 and NumPy. I have 7 single channel (gray scale) images of shape (1080, 1920). After stacking them along the third ...
- 2,518
1
vote
0
answers
126
views
Awkward Array broadcasting and linear indexing, reshaping an Awkward Array
I'm trying to use numpy-like syntax on an awkward.Array with variable sizes in the second dimension, but it's still confusing..
In numpy i have
normals.shape # (N,3)
idcs.shape # (m,k)
normals[idcs]...
- 11
4
votes
1
answer
226
views
Why does this error when converting a python list of lists to a Numpy array only occur in specific circumstances?
I have a somewhat peculiar structure of python list of lists that I need to convert to a numpy array, so far I have managed to simply get by using np.array(myarray, dtype = object), however a ...
- 95
0
votes
0
answers
64
views
Numpy slicing gives unexpected result
Does anybody have an explanation for the unexpected numpy slicing results dislplayed below ?
Unexpected behavior demo
import torch
import numpy as np
some_array = np.zeros((1, 3, 42))
chooser_mask = ...
- 838
0
votes
2
answers
136
views
Faster numpy calculations than reshaping with einsum
Consider the following in Python: A has dimension (T,), U has dimension (L,T) and G has dimension (K,T), Y is (L,L,T). My code outputs a numer1 and numer2 with dimensions (T, LK, 1), . Consider that ...
- 377
0
votes
1
answer
99
views
Most efficient way to index a numpy array by the indices of another numpy array
I have a numpy array A with shape (a, b, c), and another integer array L with shape (a, b). I want to make an array B such that B[i, j, k] = A[L[i, j],j, k] (assume shapes and values of L permit this)....
- 135
1
vote
2
answers
56
views
How to add each row of a matrix to the selected rows of another matrix?
I want to achieve the following result fast
for i in range(n):
C[index[i]] += A[i]
where C is an d * m matrix, index is an array containing n integers ranging in [0, d), and A is an n * m matrix.
...
1
vote
0
answers
37
views
ValueError: could not broadcast input array from shape (10,3) into shape (3,)
I am trying to make a N-body simulator that takes initial position and velocity and with output of position respect to time. I used create a function that produce the differential equation of velocity ...
- 11
-1
votes
1
answer
243
views
Numpy Broadcasting - Need complete understanding
I am trying to understand Numpy Broadcasting.
So I want to understand why is the below code working?
a = np.arange(4).reshape(2,2)
b = np.arange(6).reshape(3,2)
a = a[:, np.newaxis]
a + b
I mean if ...
2
votes
3
answers
92
views
Propagating true entries along axis in an array
I have to perform the operation below many times. Using numpy functions instead of loops I usually get a very good performance but I have not been able to replicate this for higher dimensional arrays. ...
- 693
0
votes
1
answer
61
views
Getting distances of points in 2D space in an array in Fortran using the concept of broadcasting (Python)
I'm new to Fortran and I already have a hard time understanding the concept of broadcasting of arrays in Python, so it is even more difficult for me to implement it in Fortran
Fortran code:
program ...
0
votes
3
answers
113
views
Efficient shift and roll in numpy without pd.Series
Consider the code below which gives the wanted output:
import numpy as np
import pandas as pd
sumvalues = 2
touchdown = 3
arr = np.array([1, 2, 3, 4, 5, 6, 7])
series = pd.Series(arr)
shifted = pd....
1
vote
1
answer
45
views
broadcasting tensor matmul over batches
how can i find dot product of each batch response and X data.
y_yhat_allBatches_matmulX_allBatches = torch.matmul(yTrue_yHat_allBatches_tensorSub, interceptXY_data_allBatches[:, :, :-1])
expected ...
- 473
2
votes
1
answer
80
views
Leverage broadcasting to make this subtraction more efficient
I have an array x of shape (N, T, d). I have two functions f and g which both take an array of shape (some_dimension, d) and return an array of shape (some_dimension, ).
I would like to compute f on ...
- 3,601
0
votes
1
answer
106
views
Finding whether each element in matrix is reciprocal of it's transpose in 3d tensor
I have a numpy matrix of shape m * n * n, meaning I have m n*n square matrices. For each matrix, I must have elements present in such a way so that the transpose indexes are reciprocal of each other. ...
2
votes
1
answer
186
views
Broadcasting with concatenate operator
numpy.broadcasting allows to perform basic operations (additions, multiplication, etc.) with arrays of different shapes (under certain conditions on these shapes). For example:
>>> a = np....
1
vote
1
answer
63
views
Nested indexing in NumPy einsum?
I'm trying to write the following expression using the einsum function in NumPy:
for j in range(100):
p[j] = 0
for i in range(100):
if i!=j:
p[j] += S[i,j]*B[T[i,j], i]
p....
- 11
2
votes
2
answers
84
views
Recursive matrix construction with numpy array issues (broadcasting?)
The following is a seemingly simple recursion for finding a hybercube matrix.
The recursion is defined as:
(Formula)
I tried to put it into code but I keep running into broadcasting issues with numpy.
...
- 23
0
votes
1
answer
110
views
Is there a dilated k-nearest neighbour solution available fast execution?
I am implementing the dilated k-nearest neighbors algorithm. The algorithm unfortunately has nested loops. The presence of loops severely hampers the execution speed.
import torch
dilation=3
nbd_size=...
- 207
0
votes
1
answer
76
views
How do I compare 2-d tensor with 1-d tensor in Pytorch?
Example of what I want to compare these two:
torch.tensor([[1,2],[1,2],[1,3]]) == torch.tensor([1,2])
I want this output:
[True, True, False]
But instead the broadcasting gets me:
tensor([[ True, ...
- 2,376
0
votes
1
answer
550
views
Pytorch broadcasting not working as expected
I am in the early stages of learning Pytorch for deep learning and have come across something I don't understand. I have written a very simple script to just make sure I fully understand the ...
- 33
0
votes
0
answers
34
views
Numba vectorize a function if only one of its inputs could have different dimensions
Given an array of points with shape (2, n) the function returns the n interpolated values.
The function should also work if points is a 1-d array (2,). I think numba vectorize could be used to solve ...
0
votes
1
answer
239
views
Fast orthogonal projection of vectors onto other vectors numpy
I have two large equal size 2D numpy arrays of cartesian vectors:
A = [[ax1, ay1, az1], [a2], [a3], ...] where a1 = [ax1, ay1, az1]
B = [[bx1, by1, bz1], [b2], [b3], ...] where b1 = [bx1, by1, bz1]
I ...
1
vote
2
answers
81
views
Numpy argmin() to find the nearest tuple
I have an array of tuples, I need to find the tuple from that array that is the closes to a given tuple(element wise), that is by the absolute value difference between each element of these two tuples....
- 314
1
vote
2
answers
76
views
NumPy row operations that depend on other rows/columns
Problem
I am trying to avoid a for loop in NumPy (which is quite messy and obviously performance prohibiting). My challenge is that operations on each row depend on other rows. That is:
I have a (very ...
- 1,195
0
votes
1
answer
70
views
How to change an array using advanced indexing and boolean array indexing without loops in numpy?
Problem:
A is a multidimensional array of two dimensions (i,j) and B is a boolean array of the same shape that I want to define according to the values of A.
I want to define B through two ...
- 673
2
votes
3
answers
417
views
Fastest way to construct sparse block matrix in python
I want to construct a matrix of shape (N,2N) in Python.
I can construct the matrix as follows
import numpy as np
N = 10 # 10,100,1000, whatever
some_vector = np.random.uniform(size=N)
some_matrix ...
- 923
0
votes
1
answer
235
views
Setting an array element with a sequence value error
Why do I get this error only for x_train? On commenting x_train out, no errors come.
---------------------------------------------------------------------------
TypeError ...
- 1
0
votes
1
answer
37
views
Rewriting numpy function to handle 2d and 3d inputs
I am trying to rewrite an numpy function such that it can deal with 2d and 3d inputs.
Consider the following code:
import numpy as np
def adstock_geometric(x: np.array, theta: np.array):
x_decayed ...
0
votes
1
answer
138
views
Why is the KNN using pytorch broadcasting is so slow?
I'm trying to find knn for grid points. This is the code for generating the grid
def grid_by(lims=[[0, 1], [0, 1]], size=[28, 28]):
"""
Creates a tensor of 2D grid points.
...
- 199
1
vote
1
answer
113
views
How to compute the moving average over 3D array with a step size?
I need to calculate a moving average over a 3D array with a step size set by me. What I am doing right now is
img = np.ones(10,10,50)
img_new = bottleneck.move.move_mean(img, window=5, axis=2)
...
- 701
0
votes
1
answer
38
views
Using Numpy Broadcast to achieve array/Matrix subtraction without looping over the indices
I was trying to perform mean clustering on my dataset. The dataset X is of the dimension (182,108,130). The mean is calculated using np.mean(X, axis = 1). The mean is of dimension (182,130). Now I'd ...
- 23
0
votes
1
answer
252
views
Numpy array broadcasting vs explicit dimension expansion space inefficiency
Why is the explicit dimension expansion so space inefficient as compared to leveraging implicit numpy broadcasting, which technically does the same thing i.e. copies the matrix over on a given ...
- 3
0
votes
1
answer
69
views
Multiplying 2D matrices to get a 3D matrix
I have two matrices A and B of dimensions (n_m, n_u) and (n_m, n). I want a 3D matrix with dimensions (n, n_m, n_u) such that the first column of B is multiplied (element-wise) with every column of A ...
0
votes
1
answer
42
views
How to use a mask to limit broadcasted operations between two numpy arrays?
I have an array like so:
data = np.array([
[[10, 10, 10],
[10, 10, 10],
[10, 10, 10]],
[[20, 20, 20],
[20, 20, 20],
[20, 20, 20]],
[[30, 30, 30],
[30, 30, 30],
...
- 6,306
0
votes
1
answer
126
views
How to replace a value by another value only in certain columns of a 3d Numpy array?
I have a 3d numpy array and a list of columns (axis = 1) in which I would like to replace all zeroes by a constant value:
My sample data is like so:
data = np.array([
[[10, 10, 10], [0, 10, 10], [...
- 6,306