yEnc Information Page for Mac OS X yEnc Info

This yEnc information is brought to you by Parker Software Development, Inc., publisher of NewsHunter3, ArcherPro, and Archer.

Contents

Introduction

This yenc Information Page is designed to provide you with useful information regarding yEnc encoding and your Mac OS X computer. In this document you will see mentioned many different programs. Some, like NewsHunter3, are published by Parker Software Development, Inc. Others are either published by other companies or made available by individuals. We provide reference to these competing applications just to make your life a little bit easier. Whichever application you select, we just want to make sure that your OS X experience is as enjoyable as possible. Especially if you have recently switched from the Windows platform and are still struggling to wrap your head around a new way of doing things. Naturally, we would prefer you use our wonderful applications.

Everything You Need to Decode Files

Spend enough time on Usenet, and you will discover that there is more than just yEnc files out there. Your newsreader most likely will not know what to do with some of them. However, there are many utilities to handle such situations. Pure-Mac has compiled an excellent list of just about every kind of encoder, decoder, assembler, and archiver available for the Macintosh OS X user. <www.pure-mac.com/compen.html>

What is yEnc?

yEnc (or yEncoding) is a method used to turn a picture, movie, audio, or other type of file it into a text file. Once the file has been converted to a text file using the yEnc method, it is ready to be posted to your favorite newsgroup. Those responsible for yEnc can be found here: <www.yenc.org>. You can also gain a little insight as to why yEnc was developed in the first place in this interesting article on yenc.org: <http://www.yenc.org/general.htm>

Why do I Have to Think about yEnc, Decoders, Encoders, MIME, Base64, and All This Other Nonsense? Why Me!

The reason why you have to think about all these different encoding methods is because they are all used in order to get around the simple fact that newsgroups will accept only letters, numbers, and a few other characters.

The “encoding” step is required because Usenet newsgroups work only with plain, old-fashioned text files. Back in the olden days, when the internet meant email, newsgroups, and file transfer protocol; in the dark times before the World Wide Web when the internet was a strange thing universities and government agencies employed to make sure communications could continue in the event of a nuclear war; in the early days when computers of different types could be trusted to understand and have in common only the most basic bits of information, it was decided that the letters “A” through “Z” and the numerals “0” through “9” would be sufficient to convey all forms of communications between all the different kinds of computers. And even then there were incompatibilities. Surely nothing beats the written word, right? In any event, the various ways of encoding stuff worked very well. It was like putting something in a box. Everyone knows how to pick up a box and give it to someone else. Once the recipient gains possession of the box, it is up to them to figure out how to open it to access the goodies inside.

The Digital Decoder Ring

Some time in 1975 someone loaded a picture into their computer using what was then one of those new-fangled things from called a “scanner”. But how could this picture file be sent to a friend using an email system or a newsgroup system that understands only letters and numbers? One of the first such solutions was to use the Unix-to-Unix Encoding method (UUE). This is probably the best, most compatible way to send pictures, audio, video, and other files (or as we call them today, “binaries”) to a person using a different computer, operating system, or even using a different network.

What the “encoding” methods do is open the “binary” file, read in a byte, translate it into a character, write that character to a text file, read in another byte, and so on until the End of File (EOF) is reached. The resulting text file can then be sent through the old-fashioned Usenet and Mail systems. When you receive such a file, all you need to do is take that text file and run it through the “decoding” method which does the exact opposite. The result: a viewable picture for example. It's all very similar to a decoder ring.

This was all well and good in the olden times. If you were on Usenet and reading newsgroups, you were probably familiar with how to handle such files. You may have even created your own encoding schemes and shared them with friends. However, back in the good old days everyone used UUE. It was well documented, was very robust, and formally spec'ed out, gated, and accepted by the computing universe.

The World Takes Notice of Usenet Newsgroups

Once the rest of the world gained access to the internet, and Usenet specifically, things changed rapidly. Suddenly people were posting all kinds of files to newsgroups without regard to the community in general. What evolved was a series specifications that allowed for all kinds of files to be sent through the usenet newsgroup system. Of course, regardless of the type of file being “posted” or the method in which it was “encoded”, it was eventually represented as a plain text file.

Soon people began to expect a certain amount of intelligence from their newsreading programs. They expected a newsreader to figure out what kind of file was “encoded” and to decode it automatically. Eventually newsreaders evolved to meet these needs to varying degrees of success. For example, a person running Windows might post a file that can only be opened by a specific application that ran only on the Windows operating system. Sure, the file would be “encoded” properly and would sail through all the various computers without ill effect, but if the person downloading the file from the newsgroup was running IRIX on their Silicon Graphics O2 or OS X on their G5 Macintosh, they would still be unable to use the resulting file once it had been decoded.

That’s one of the reasons why there are things called Industry Standards. These standards ensure that the file will be usable regardless of the type of computer the person downloading the article happens to be using. Some popular standards are JPEG for still images, MPEG for motion pictures, and MP3 for audio files. If you save your files in one of these formats BEFORE your encode it for posting, you can be pretty sure that everyone on the Usenet will be able to view your file.

What’s the Big Deal About yEnc?

Basically yEnc is an encoding step that results in smaller files. That's about it. There’s no earth shattering consequences for most users. yEnc, which grew organically into acceptance rather than being formally gated, is now one of the most the most popular methods of encoding files for posting in newsgroups. Despite its popularity, yEnc does have its detractors. You can read their viewpoint on their web site: <http://www.faerber.muc.de/temp/20020304-yenc-harmful.html>. You can also read about proposals to replace yEnc with an encoding method called MZIP Compression and is based on ideas described in RFC 1951.

Can my Mac Decode yEnc Files?

Of course your Mac can decode yEnc files. Some programs will decode yEnc files for you, others will employ the use of a yEnc decoder that you select yourself, and still others will display the “yEnc” file right there in your newsreader’s article viewer. In each case, decoding the file is fairly simple. Let’s talk about each method.

Programs that Decode yEnc Files for You

Programs like Diiva, Halime, PictureSnooper, Thunderbird, Thoth (No longer available), MT-Newswatcher, Hogwasher, NewsHunter 3 provide for built-in decoding of yEnc files. However, some users like to have the option of selecting an application for decoding yEnc files.

Programs that Let You Select a yEnc Decoder

Programs like NewsHunter 2 allow you to select an application to decode yEnc and other types of files. This gives you the flexibility to use a decoder utility that is, for example, faster, or that provides unique options not available with a built-in decoder. yEncTZ is a good app for decoding yEnc files. Another fine application for decoding yEnc files is Decoder. Also available is yencode which provides command line control for the encoding and decoding of yEnc files.

Programs that Process Files “Offline”

Some programs will also provide an option to disable decoding of files completely. This kind of option will cause your newsreader to simply download articles into a spool file. You can then deal with them “offline”. Downloading a newsgroup for “offline” reading is handy for those who have to pay large amounts of money to access the internet. Since you are not wasting any time decoding files, you spend less time “online” and thus incur fewer access charges. MT-Newswatcher and NewsHunter 3 support this type of downloading.

Programs that Display yEnc Files in Textual Format

Some programs will not recognize the yEnc file and will display it as if it where just another article posted to the newsgroup. The problem with this is that few people will be interested in “reading” a yEnc file. Here is what the beginning of a yEnc file might look like.

=ybegin line=128 size=29034 name=smudges_the_cat.jpg
))=J*:tpsp*+,+*)<#oê**ww

This is then followed by lots of unreadable junk. Many programs, especially older programs will download yEnc files but will have no idea what to do with them. In the case above, you could simply select all the text once it has finished loading, copy it to the clipboard, paste it into a text file, and drag the resulting text file onto your yEnc decoders icon (yEncTZ for example). Or, you might use yencode and a shell window to decode the resulting text file.

Regardless of the application you use, you should have no problem decoding yEnc files. Even if your newsreader does not support yEnc, you can still copy and paste the article into a text file and decode it manually. And if your newsreader lets you disable decoding of files, you can save money because of the reduced time spent online. You decode the files later on, after you have logged off.

Why Do Some Files End with Number Like .000, .001, .002? (Segmented yEnc Files)

Ever see things like this in your favorite newsgroups?

example_file "Cats_with_Cheese.mpg.001" yEnc (1/1) 500KB [002 of 089]
example_file "Cats_with_Cheese.mpg.002" yEnc (1/1) 500KB [003 of 089]
example_file "Cats_with_Cheese.mpg.003" yEnc (1/1) 500KB [004 of 089]

These are what many people call “segmented” files. This is a term used to describe the way in which large files are posted to Usenet newsgroups. Again, this goes back to olden times when disk space was incredibly expensive and administrators limited the size of articles to keep costs down and to keep users from hogging up all the disk space with gigantic posts of things like pictures of the Starship Enterprise and other important stuff. Over the years, this arbitrary limit has been raised and most systems will let you post files as large as 1 MB. Still, to be sure that as many people get their posts as possible, authors (the name we give to people who post stuff to Usenet newsgroups.) will segment files into 500KB pieces.

This 500KB size is often used so that the segments “propagate” to as many machines as possible. For example, if your news service provider allows you to post files as large as 1MB, those segments will be rejected by news servers that limit article sizes to 500KB or less. In other words, your large article will not be propagated to machines that accept only small articles. Millions of people might not see your post!

This situation is fluid and it is now common to see segment sizes of 1MB propagate very well. Most news service providers are recognizing the growth of binary posting in newsgroups and have raised their limits accordingly.

So, not only are the files yEncoded but they are segmented as well. This seems like a real problem and vexes many new users who wish they could “just watch the movie” without having to jump though hoops to do it.

Some files end with numbers so that when you receive them on your end, you know the order in which to assemble them. Oh, man! Now you need an assembler to put all the segments back together.

Not to be Confused with MIME Files

Some people will refer to segmented files as multi-part files or multi-part posts. This is all well and good until you try to mix in the subject of MIME (Multipurpose Internet Mail Extensions). This is because an article posted using a MIME encoder can actually contain multiple files in the same article. Each file is represented by an individual “part” within the MIME encoded file.

A large file posted across several articles is a segmented file. A file or files that are posted in a single article using a MIME encoder would be better described as a multi-part article or multi-part post. It is, however, conceivable that a MIME encoded article containing several pictures (or “files” as this fine Desktop image of Dr. Evil might call them.) can be segmented as well. People who post stuff like this suck.

Most good applications can decode MIME files or better yet, allow you to select an application of your choosing to decode such files.

How Do I Put These yEncoded, Segmented Files Back Together?

OK. So now you've managed to beat up your newsreader enough and it begins to download and decode these segmented yEnc files. But now what? If you use an application like NewsHunter, you see bunch of seemingly useless files in your Decoded Files folder with names like Cats_with_Cheese.mpg.001, Cats_with_Cheese.mpg.002, and so on. In fact, there will likely be dozens or even hundreds of these stupid files. Some clever newbies will remove the .001 from the first file and end up with a file named “Cats_with_Cheese.mpg”. If double-clicked, the file will open in your QuickTime player and you will enjoy a brief snippet of a movie. Try doing it with the remainder of the files, and you end up with nothing. The trick will only work with the first segment.

With Split&Concat of Course

What is needed is a program to assemble all of these segments into one big, playable file. The best program for this job is Split&Concat. All you do is launch Split&Concat, click the Concat button, and select the first segment. In our example, the file named “Cats_with_Cheese.mpg.001” would be selected. Split&Concat does the rest. It will be able to assemble the file even if some of the segments are missing. The usefulness of this little app cannot be underestimated. Plenty of opportunities to donate some cash to author are presented.

I Go Through All This and Now the File Won't Play!

Finally! You’ve queued up a gazillion files, set your newsreader to download the files, and have logged out of your newsreader account and back into your normal work account. (You do realize that with OS X you can create an account that you can use just for your Usenet activities, right? Click on the User Menu in the upper left hand side of the menu bar to log out of one account and into another. Your newsreader will continue to operate in the background account while you work in your regular account.) To create accounts open the System Preferences and click on the Accounts icon.

You switch accounts , run everything through Split&Concat, and end up with a file that QuickTime Player refuses to open. A frustrating experience to say the least.

Try MPlayer 2 OS X

Well, chances are that MPlayer will be able to open that file. You can and should download a copy of MPlayer. You will find it most useful.

Other Yenc Resources

This document is by no means the complete guide to all things yEnc. There are many other resources available on the internet dealing with the yEnc encoding method. Here is a short list of some of them.

Conclusion

This document has attempted to assist people in downloading, decoding, assembling, and opening files that have been encoded using the yEnc specification. An attempt has been made to address concerns that are specific to the Mac OS X operating system. If you feel we have omitted any information, please feel free to let us know.

Contact

You can contact us at support@parkerosftware.com

Disclaimer

Note: This document is provided for informational purposes only. Please refer to this page for other legal stuff.