I see lots of references saying things like
A tensor is a multidimensional or N-way array
But others that say things like
it should be remarked that other mathematical entities occur in physics that, like tensors, generally consist of multi-dimensional arrays of numbers, or functions, but that are NOT tensors. Most noteworthy are objects called spinors.
Are the terms “tensor” and “multidimensional array” interchangeable? Or is it more like “all squares are rectangles but not all rectangles are squares”?
Tensor : Multidimensional array :: Linear transformation : Matrix.
The short of it is, tensors and multidimensional arrays are different types of object; the first is a type of function, the second is a data structure suitable for representing a tensor in a coordinate system.
In the sense you’re asking, mathematicians usually define a “tensor” to be a multilinear function: a function of several vector variables that is “linear in each variable separately”. A “tensor field” is a “tensor-valued function”.
In more detail, if M is a smooth manifold, there are a tangent bundle TM, a cotangent bundle T∗M, and “tensor bundles” obtained by taking tensor products of these bundles. A tensor field on M is a section of a tensor bundle. (The answers to the question linked by John Mangual flesh out the details.)
Physicists call tensor fields “tensors”. I’ll (continue to) use mathematician-speak.
In a coordinate chart (i.e., a piece of M diffeomorphic to an open set in a Euclidean space, together with a coordinate system), the tangent and cotangent bundles are trivialized by coordinate vectors and coordinate 1-forms, and a tensor field is represented as a multidimensional array of functions.
The “transformation rules” for multidimensional arrays amount to the chain rule for vector fields and 1-forms, i.e., to transition functions for tensors corresponding to changes of coordinates in M.