How do I convert an ArcGIS (10.3) spatial weight matrix (SWM) file to a numpy array directly?
swm -->(swm to table)--> table -->(table to numpyarray)--> numpyarray
is very slow with 16.000.000 entries. There must be a way to directly (or at least faster) convert a SWM to numpy array, is there not?
You can use Python to read the matrix file even though it's binary. numpy is capable of reading this binary file using the fromfile method.
matrix = r"C:\temp\matrix.swm"
header = open(matrix, 'rb').readline().strip().split(";")
arr = np.fromfile(open(matrix, 'rb'), '<l')
arr
array([ 1397900630, 1078873929, 825110577, ..., -1717986919,
4180377, 0])
Another example of reading the file into numpy array:
How to convert GeoDa .GAL or ArcMap .SWM file to Matlab (.dat)?
I have been fooling around for a bit. The below code takes an .swm-file as input, and outputs a numpy matrix (saved as a .npy-file) where the relation between feature 1 and 7 can be found in row 1 (index 0) and column 7 (index 6).
nhs is a list that contains the relationships from 'masterID' to each feature in the list. e.g. [500,300,123,689] means that a relationship from masterID to feature 500,300,123,689 is stored in the weights-list.
Funny enough, the .npy file is smaller than ESRI's .swm file, even though all relationships are stored, wheareas the .swm file only stores unique relationships (e.g. 1-->7 is stored, so 7-->1 is not stored).
#####import
import arcpy
import sys
import numpy as np
np.set_printoptions(threshold=10)
##### import parameters from Arcgis GUI
SWM_file = arcpy.GetParameterAsText(0)
output_folder = arcpy.GetParameter(1)
##### Open the SWM_file
swm_open = open(SWM_file,"rb")
header = swm_open.readline().strip().split(";")
obs,row = np.fromfile(swm_open,"<l",count=2)
##### create np array that is to be filled with the weights
arr = np.zeros((obs,obs))
##### Get the weights from the swm file and fill the arr
for i in xrange(obs):
masterID,nn = np.fromfile(swm_open,"<l",count=2)
if nn != 0:
nhs = np.fromfile(swm_open,"<l",count=nn)
weights = np.fromfile(swm_open,"<d",count=nn)
sumunstandard = np.fromfile(swm_open,"<d",count=1)
for z in range(len(nhs)):
row = masterID - 1
col = nhs[z] - 1
arr[row][col] = weights[z]
##### set the diagonal equal to 1
for i in range(arr.shape[0]):
arr[i][i] = 1.0 #set the diagonal to 1
##### set missing value from A to B equal to the value B to A
for i in range(arr.shape[0]):
for z in range(arr.shape[1]):
if arr[i][z] == 0.0:
arr[i][z] = arr[z][i]
elif arr[z][i] == 0.0:
arr[z][i] = arr[i][z]
##### Save the file
outfile = str(output_folder) + "\NumpySWM.npy"
np.save(outfile,arr=arr)
##### Terminate script
sys.exit()