Getting an error when trying to fit factor loadings to original data

[gaardhus]

Describe the bug
I'm doing factor analysis on a matrix with the dimensions 17x153, so I have more columns than rows. Normally, I would expect a PCA to return the same number of components as variables, but in my instance it only returns 17. I get the correct factor loadings for the 17 components, however when I try to fit it to the original data - in order to see how my original observations are distributed on the components - I get an error, and values that do not correspond with my tests.

C:\Users\UserName\anaconda3\envs\DS_env\lib\site-packages\factor_analyzer\factor_analyzer.py:740: UserWarning: Unable to calculate the factor score weights; factor loadings used instead: Singular matrix
  'factor loadings used instead: {}'.format(error))

Desktop (please complete the following information):

• OS: Windows 10
• Browser: Firefox

Additional context
Python version: 3.7.8

[jbiggsets]

Thanks for submitting this issue.

The way we're calculating the principal factors currently is using the randomized_svd() function from scikit-learn. This is a lot more efficient and resolves some of the intrinsic sign indeterminacy problems with SVD. However, it has the drawback that the number of factors is limited to K, where K = min(M, N). In your case, the number of rows is less than the number of columns, so it's constraining K to the number of rows.

An alternative is to use scipy's svd() function with full_matrix=True (the default). The drawback here that it's less efficient with larger data sets and, at least as far as I understand, does not have an easy way to resolve the sign indeterminacy the way randomized_svd() does when flip_sign=True. (If I'm wrong about that or you're aware of a work-around here, please let me know!)

Assuming I'm not missing anything, one way to resolve this might just be to implement both options and give users the ability to decide? @desilinguist, if you have thoughts, let me know.

[desilinguist]

I like the option of giving the users the choice. Is that a lot of work?

[jbiggsets]

I like the option of giving the users the choice. Is that a lot of work?

No, I think it should be pretty easy to implement. I can work on it with this next release, if we're aiming for mid-April.

[desilinguist]

Fantastic! Let's go for it.

[jbiggsets]

The more I look into this, the more I think that we shouldn't actually permit this use case. I still think we can allow users to choose between randomized_svd() and scipy.linalg.svd(), but scikit-learn's PCA() estimator also prevents users from trying to do PCA when the number of components is set higher than min(n_samples, n_features).

import numpy as np
from sklearn.decomposition import PCA

X = np.random.randn(5000).reshape(20, 250)

n = 21
for solver in ['full', 'randomized', 'arpack']:
    try:
        pca = PCA(n_components=n, svd_solver=solver)
        pca.fit(X)
    except ValueError as error:
        print(error); pass

Output:

n_components=21 must be between 0 and min(n_samples, n_features)=20 with svd_solver='full'
n_components=21 must be between 1 and min(n_samples, n_features)=20 with svd_solver='randomized'
n_components=21 must be between 1 and min(n_samples, n_features)=20 with svd_solver='arpack'

[desilinguist]

Yeah, let's follow their lead and do the same then?

[jbiggsets]

Yeah, let's follow their lead and do the same then?

Yeah, I think that makes the most sense.