XMedia Encode (http://www.xmedia-recode.de/download.html
for latest, videohelp for older versions (mine is 188.8.131.52 latest is 184.108.40.206)
MKVmerge (for merging subs to your finished MKV file)
TsMuxer/Sup2Sub for extracting BD subs and converting to Subs
1. Download and install XMedia Encode (doesn't matter if it's original or portable, and do remember to set language to English)
2. Open XMedia Encode. You should come across in the second menu bar a 'DVD' and 'Movie'. Since the DVD part is for Video_TS and VOB files, and Blu-Ray files are only .m2ts (I think .m4v or .mpi as well, I got to go check on those later), you would instead choose 'Movie'. Next, open the file and search for the Blu-Ray that you're doing. Go until you find 'Stream' and open that, then choose what .m2ts files you want.
2b. Highlight one of the m2ts files before going on.
3. Go to 'Format' and copy what you see here. This tutorial is mainly for converting to MKV, but you can choose OGM, MP4, VOB, WMV, AVI, etc.
4. Go to 'Video' and General. Copy what you see here but don't copy the bitrate. You'll know why in the next step.
5. Open Options -> Bitrate Calculator. This is the place where you can change the bitrate of the audio and/or the video, which will change the file size, and vice versa. I recommend messing with the file size first to get your desired bitrate. As for the Rate Control Mode in the Video Codec, there's four to choose from: Average bitrate, 2-pass bitrate (Handbrake), Constant Quality (I guess also Handbrake, although will disable changing bitrate and file size), and Constant Quantizer (no fucking idea wtf this is). I choose Average bitrate. Press OK when done.
6. Leave Deblocking, Partition Marcoblocks, Rate Control, Motion Estimation, Slicing, BluRay, and Quantizers as is, since I have NO idea what they are. Select B-Frame and make Number of B-Frames to 0. Next, select Extra and copy the following:
7. Now go to 'Audio Track 1' (and 2 if you're doing dual-audio). All you need to focus on is the first three and bitrate.
8. If you're doing a Dual-Audio anime, I would skip 'Subtitles' because XMedia will HARDCODE the subs. That is why you will need TSmuxer to extract the subs and Sup2Sub to convert them into .sub/.idx files.
9. Filters/Preview, Resolution (and maybe Crop, just in case the video is cropped). The default Blu-Ray show will always be 1920x1024/1080, and 16:9. You may change the hxw to whatever you like, or just check Keep Aspect Ratio and change the width, which will automatically change the height. You could also change the start and end times, but with this program it ain't good so don't bother.
10. Chapters, Tags. You're on your own...
11. If one of your m2ts files is still highlighted, click 'Add Job' then go to 'Jobs'. It'll preview everything that you did before you begin to encode.
12. If you're doing multiple files, just highlight another file and repeat 3-11. Last, Save to folder, choose a destination that you're saving to, and click Encode!
Encoding a single BD ep could take 4 hours up to a day, and that's even without Internet or other programs that eat up RAM, but luckily while encoding you can pause it, surf the Internet or do what you gotta do, and then resume when you're done.