Singly and doubly charged Higgs bosons in the Higgs triplet model mediate the lepton flavor violating (LFV) decays tau -> lll and mu -> e gamma. The lepton flavor violating decay rates are proportional to products of two triplet Yukawa couplings (h(ij)) which can be expressed in terms of the parameters of the neutrino mass matrix and an unknown triplet vacuum expectation value. We determine the parameter space of the neutrino mass matrix in which a signal for tau -> lll and/or mu -> e gamma is possible at ongoing and planned experiments. The conditions for respecting the stringent upper limit for mu -> eee are studied in detail, with emphasis given to the possibility of |h(ee)|similar or equal to 0, which can only be realized if Majorana phases are present.