Usually when you're seeing that message its because the mods or texture back being loaded have exceeded the amount of memory allocated to the Java client. If what I'm saying makes no sense, then that means you're still running stock settings, and will need to bump that up a bit. If you're using the vanilla minecraft launcher, select the Launch Options tab (it defaults to news at the top). The select your version of minecraft you wish to edit from the list below. JVM arguments should already be enabled, but might have a switch that says something like -Xmx1G or -Xmx1024m. Changing this switch allows you to let Java use more memory if it needs to. Try changing it to -Xmx4G or -Xmx8G if you have the system memory for it.
If you're using the twitch client go to your modpack you're using, select it, and select the ... button on the far right near the top. From the dropdown there, select Profile Options. Under Memory Settings, uncheck Use System Setting, and drag that slider up into or above the recommended bars, again depending on how much ram you can afford to allocate to java.