Direction of arrival (DOA) estimation via directivity pattern analysis (DPA) has been proposed for years, in order to locate signal sources. It uses directional nulls existing in directivity patterns to approximate angles. However, traditional directivity pattern analysis often causes biased estimation, which is usually derived from ambiguous patterns. In this study, we convert the DOA problem from the signal domain into the visual domain, so that pattern analyses and recognition techniques are applicable. A novel architecture, composed of adaptive cascaded separators and a neural network, is presented to minimize the effects of obscure directional nulls. We also employ an adaptive algorithm for collecting the refined information generated by the neural network and updating the separators automatically. The experimental results show that this system is less susceptible to the effects of inappropriate patterns than other systems. Simulations were performed to compare results between the conventional approaches and our proposed method.