Equivalent of numpy.where or numpy.nonzero

[tsoernes]

Is there an efficient way to get the indecies of non-zero or true elements?

[jturner314]

This is an example that provides solutions for both problems:

#[macro_use]
extern crate ndarray;

fn main() {
    let numbers = array![[0, 1, 0], [2, 0, 3]];
    let nonzero: Vec<_> = numbers
        .indexed_iter()
        .filter_map(|(index, &item)| if item != 0 { Some(index) } else { None })
        .collect();
    assert_eq!(nonzero, vec![(0, 1), (1, 0), (1, 2)]);

    let bools = array![[false, true, false], [true, false, true]];
    let nonzero: Vec<_> = bools
        .indexed_iter()
        .filter_map(|(index, &item)| if item { Some(index) } else { None })
        .collect();
    assert_eq!(nonzero, vec![(0, 1), (1, 0), (1, 2)]);
}

It's probably not particularly fast, especially for arrays that don't have a row-major memory layout, but it works.

Out of curiosity, what's your use-case for this? There may be a better way to solve your problem than using a list of indices.

[tsoernes]

The use is (partially) to select a non-zero entry at random and set it to zero, as part of a simulator. where and nonzero are pretty commonly used functions in numpy, it would be nice with a fast, built in function. Thanks anyhow

[jturner314]

Yeah, I'd like to add something like where or nonzero to ndarray. I was curious about your use-case because where and nonzero didn't seem very useful without support for slicing with index arrays (which I'd also like to add to ndarray), but using them for random selection make sense.

[bluss]

I wonder if this problem would be more efficient if we had something like a bitmap. Storing indices just sounds so inefficient and an ndimensional bitmap could be a faster (just a wild idea?) way to represent a set of locations in an array.

[tsoernes]

I have also found that masks can often substitute for lists of indexes. But there are certainly use cases where this is not so. For example, I often have the need to get the values at a set of non-continuous indexes in a large matrix. If the matrix is large compared to the number of values you want to extract; e.g. you don't want to multiply a million zeroes to leave only 10 entries as-is. See e.g. the latter half of this file: https://github.com/tsoernes/rustdca/blob/master/src/gridfuncs.rs where any reference to neigh(bor) or ch(annel) is an array of indexes. Most of these algorithms cannot be feasibly be converted to the use of bitmasks. np.where is used all over in machine learning.

[bluss]

So, a sparse bitmask maybe?

[tsoernes]

Wouldnt a sparse bitmask be implemented by storing a list of indecies?

…

On Fri, Nov 23, 2018, 14:31 bluss ***@***.*** wrote: So, a sparse bitmask maybe? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#466 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGd9xJ2lSqazalCOJkSImeP5dY_BRSXzks5ux_i2gaJpZM4U08WJ> .

[bluss]

I guess a good bitmask can handle both scenarios. There's an alternative to list of indices here https://github.com/brettwooldridge/SparseBitSet/blob/master/SparseBitSet.pdf and then there's roaring, and I'm not sure exactly what data structure it is using. https://github.com/RoaringBitmap/RoaringBitmap