$$\require{cancel}$$
$\mathbf{H} = -\frac{1}{2h^2} \begin{bmatrix} -2 & 1 \\ 1 & -2 & \ddots \\ & \ddots & \ddots & 1 \\ & & 1 & -2\end{bmatrix} + \begin{bmatrix}V_0 \\ & V_1 \\& & \ddots \\ & & & V_{N-1}\end{bmatrix}.$
Hence, if there are $$N$$ diagonalization points, the Hamiltonian matrix has a total of $$N^{2}$$ entries, but only $$O(N)$$ of these entries are non-zero.