Open folder in alternate stream. Four kinds of NTFS metadata

CIOs spend a lot of time and resources on projects related to analytical processing of sales information and other standard business data. At the same time, dashboards are created for managers, displaying the company's performance indicators and helping to make forecasts for the future. Such systems bring considerable business value, but in fact, the opportunities that open up thanks to them are only a small part of what can be done with the data available to the organization, says Krishna Nathan, CIO of S&P Global (formerly McGraw Hill Financial), which deals with the management of credit ratings, as well as providing consulting and analytical services for the stock market. Under Nathan's leadership, a new enterprise-wide data processing system was designed and implemented, and a strategy is being implemented to accelerate business growth and create new offerings for customers.

Some companies are beginning to collect additional data - called alternative, unconventional, or orthogonal. This is a new direction for now, but CIOs should be exploring the technology today. After all, very soon alternative data will become a must-have tool for many companies.

However, do not rush to hire the next expensive specialists. Let's see what, in fact, we are talking about.

What is "alternative data"

Nathan defines alternative data in this way: it is data that comes from non-traditional sources, and analysis of it can extract useful information in addition to what you get in the usual way.

Let's say you have commercial network and you intend to open new shop in another city. Usually, this decision is based on the performance of your stores in a particular city and other metropolitan areas.

The source of alternative data here could be images of supermarket parking lots taken over several months - the occupancy rates of parking lots can be correlated with sales. As well as information about pedestrian traffic in the area where the store is planned to be opened. By combining this knowledge, you can learn something new that will help you in your business.

S&P Global also provides analytics services for commodity exchanges, and the CIO has to constantly think about how to help alternative sources data offer customers more information on how to combine different information to give customers information that they would not be able to get anywhere else.

Let's say S&P Global has information that an oil refinery in Rotterdam can produce 100,000 barrels of petroleum products per day. But due to a shortage of supplies, it processes about 70 thousand barrels, that is, free capacities for another 30 thousand are available. What will happen after an oil tanker with 30 thousand barrels entered the port? “If the report on the available capacity of the refinery is a week old, then we will not know that the oil has just been unloaded,” explains Nathan. - That is, traditional data is outdated. And this is where such a source of alternative data as satellite imagery comes in handy. If we analyze satellite imagery along with other sources, we get a more accurate picture of stocks and production in almost real time. "

Alternative Data and CIO

Even if you do not have ready-made use cases, get acquainted with new technologies. Plan systems that will combine multiple data sources for analysis. Learn to manage the data delivery chain, protect it, take into account the rights of use. And hire the right staff - you need experienced data scientists who can analyze it and extract useful information.

For quick launch of a project in the field of alternative data, you can use a ready-made solution. This is what S&P Global did when Platts, a subsidiary of the company, acquired cFlow, a satellite imagery interpretation toolkit. CFlow offers data visualization tools that allow you to track changes in trade flows along the routes of vessels, provides information about the volume and nature of tanker cargo.

Convince management that it's time to invest in alternative data - buying existing solutions or building your own. Some of your alternative data projects will pay off, but many will not work. Well, if alternative data brings really valuable information, use it to raise funds for new projects.

- Martha Heller. What is ‘alternative data’ and how can you use it? CIO. JAN 3, 2017

Windows operating systems are endowed with two little-known features hiding data: streams NTFS data(also known as alternate data streams) and access to a list of resources based on Access-based Enumeration (ABE) permissions. Alternate Data Streams provide the ability to add hidden information such as file information to a file. You probably won't need to use hidden streams of data, but attackers can use this technology against you, so you should be aware of it and how it might work.

As for the ABE method, it can add to your arsenal. This method allows you to do invisible folders and shared resource files for those users who do not have permission to access them.

Here's what you need to know about these products.

Rivers that feed the sea of ​​data

Alternate Data Streams is a function of the file NTFS systems... They were once given to Windows NT 3.1 to enable NT and Macintosh users to share files.

An NTFS file is made up of data streams. This is the standard data stream $ DATA, and possibly one or more alternate data streams. Anyone with the necessary permissions to work on the file can see the existing $ DATA data stream, open it, and read and write data to that stream.

The alternate data stream is Additional Information or files that a user or application can attach to an NTFS file. Only the user who created it knows about the existence of an alternative data stream. Typically, users are unaware of whether an alternate data stream is attached to the file; the point is that neither the contents of this stream nor its name are visible. In addition, there is no way to see the change in file size.

There are many ways to use alternate data streams. V Windows system these streams are used to store summary data of documents generated by applications that are not included Microsoft Office such as plain text files (.txt). You can enter summary information such as title, subject, and author information on the Summary tab of the file's Properties dialog box. This summary data is stored in the SummaryInformation alternate data stream.

Windows applications such as Encrypting File System (EFS) and Windows Explorer use alternate data streams to attach file-specific data to files stored on NTFS-formatted drives. EFS uses alternate data streams to attach encoding and decoding information to encrypted files to enable decentralized encryption and decryption through EFS.

Implemented in Windows XP Service Pack 2 (SP2) Microsoft application Internet Explorer(IE) uses the Security.Zone alternate data stream to provide security zone classification for files written to an NTFS volume. As a result, IE has the ability to block exploitative attacks that might occur in situations where a user downloads malicious code from an insecure Internet zone and stores this code on the local hard disk. IE categorizes locally stored content in the Local Machine security zone, which provides more privileges than the Internet security zone. XP SP2 always checks the Security.Zone data stream before allowing the loaded code to take any action on the local system.

Channel for injecting malicious code

Alternate Data Streams become noteworthy and dangerous because their names and contents do not appear in Windows Explorer. Therefore, organizers of various types of attacks consider such streams to be a convenient means of hiding data or malicious code that has entered the system. An example of using these threads is the worm [email protected] Hackers used an alternate data stream to attach multiple scripts in the language to an existing ODBC .ini file Visual basic(VB).

When activated, the worm creates account with administrative powers and sends itself to the addresses that it itself finds in the address book of Microsoft Outlook.

Another danger lies in the fact that the disk space allocated for alternate data streams is not displayed in the data about sizes (files) and about unallocated disk space of Windows Explorer. A hacker can use alternative data streams to fill the disk space of the file server, and the administrator will only have to puzzle over trying to get to the bottom of the problem. In addition, it should be noted that the command line utility Dir does not take alternate data streams into account when calculating size data (files and folders). Currently, there is only one Microsoft tool that can account for alternate data streams when calculating sizes: the Chkdsk utility.

Adding a new stream

Anyone who has write access to an NTFS file can use the usual commands operating system to attach an alternate data stream to the file. For example, the following command creates an alternate data stream mystream, concatenates mystream to a file named file.txt, and stores the phrase "top secret" in mystream.

echo top secret> file.txt: mystream

You can view the contents of mystream stream using the command

As noted above, you can add to alternate data streams executable files... So, it is possible to add blind copy Windows calculator(calc.exe) to the file.txt file. To do this, you just need to enter the command

type calc.exe> ​​file.txt: calc.exe

To launch the hidden calculator, enter the command

start .file.txt: calc.exe

You can see for yourself that alternative data streams and their contents are not displayed in instrumental Microsoft tools... Open up Windows program Explorer and view the properties of the file.txt file in it. In fact, the file size is 112 KB (this is how much space is occupied by the embedded calc.exe file) - but the program will show the file size as 0 KB: there is no information about the embedded file in the $ DATA data stream, and Windows application Explorer does not have the ability to read information from the alternate data stream.

It is clear that there are many threats associated with alternative data streams, especially in networks where the work on issuing permissions to access NTFS resources is not given due attention and strict control of access to Windows servers... There is a simple security mechanism that can discourage hackers from exploiting alternate data streams, the NTFS access control system. If attackers do not have permission to write data to the file, they will not be able to create alternate data streams and attach them to this file.

Identifying changes

If you get the feeling that hackers managed to get around the barrier set permissions, use one of the developed so far tools discovery of the content of alternate data streams. System integrity checkers such as Tripwire Enterprise and Tripwire for Servers detect all NTFS file system changes that have taken place in Windows, including adding or modifying data stream content.

Sysinternal's Streams is a freeware command line utility that determines the names of alternate data streams attached to files. Figure 1 shows how to use the Streams utility to view the name of the calc.exe data stream, which we added to file.txt earlier. This utility can be downloaded from http://www.sysinternals.com/utilities/streams.html.

Another easy way to detect an alternate data stream is with using Windows Explorer copy the suspicious file to the drive from file system other than NTFS (say, a FAT drive). Other file systems are not equipped with alternate data stream facilities. Therefore, if you try to copy an NTFS file with alternate data streams attached to place it on a different file system, NTFS will display a warning similar to the one shown in Figure 2. But be aware that if you copy this file from the Command Prompt window using the Copy command , Windows will copy it to something other than NTFS file system and without warning will delete the data stream.

Hiding shared resources with with ABE

ABE is additional function file sharing layer that Microsoft first implemented in a package Windows Server 2003 SP1. It can be used on any Windows share, regardless of the file system where the shared data is stored. ABE allows administrators to hide folders and files stored on shared resources from users who do not have appropriate NTFS-level permissions to access them. In other words, it's about folder-level security.

In cases where ABE is not applied, users connecting to a shared directory see all files and folders located on the share, including those that they do not have permission to read and those that are blocked from access to them. When a user tries to open a file or folder that he is not allowed to access, the system displays an error message explaining that access is denied. These error messages can be confusing to users, so enabling ABE can reduce the workload of the help desk.

However, using ABE has its drawbacks. Before returning the list of objects in the folder to the client connecting to the share, the server must check all ACLs for those objects so that it can determine what data to return. This can result in a significant decrease in system performance, especially when accessing shared resources containing many objects.

ABE tools are useful, for example, to configure shared resources in user home directories. Instead of creating a hidden public share for each user's home directory, you can create one shared resource containing the home directories of all users in the root home directory. Users will connect to this root directory and you can use ABE as well as NTFS permissions to control the visibility of all users' home directories.

Activating the ABE function

This function applies the new SHI1005_FLAGS_ENFORCE_NAMESPACE_ ACCESS share level flag; at the time these lines are written, it is implemented only in Windows packages 2003 SP1 and Release 2 (R2). This flag means that you are applying the ABE function to one of the folders.

Property extensions can be used to set the flag Windows folders Explorer or the abecmd.exe command line tool. Microsoft distributes the ABE Explorer extension and abecmd.exe in the ABE installation package, which is additional module for Windows platforms Server 2003 SP1. The installation package can be downloaded from the Microsoft site at http://www.microsoft.com/downloads/details.aspx?FamilyId=04A563D 9-78D9-4342-A485-B030AC442084. Since ABE is a server-side extension, it can be used no matter what Windows version installed on the client.

After installing ABE tools on the server, you can set this flag for a particular folder. Right-click the folder, select Properties, go to the Access-based Enumeration tab, and select the Enable access-based enumeration on this shared folder check box, as Figure 3 shows. To apply the ABE function to all shared resources on the system, select the Apply flag this folder "s setting to all existing shared folders on this computer.

The second method is to use the abecmd.exe command line tool. To apply the ABE function to the shareddocs share, enter the following command:

abecmd / enable shareddocs

You can use the / all option to enable ABE on all available resources, and / disable to disable ABE.

Access control

ABE is a simple tool that allows you to restrict user permissions to access only those files that they need to work. Users can easily find required files because they don't have to wade through irrelevant folders, and they don't bother support with questions about why files they don't have permission to open won't open.

To protect against hackers using alternate data streams, administrators should monitor public resource access control settings and use one of the utilities I've described to identify hidden alternate data streams as well as NTFS changes.

Jean de Klerk(declercq @hp .com) is a Security Office employee at Hewlett-Packard. Engaged in identity management and security Microsoft products... author Windows books Server 2003 Security Infrastructures (Digital Press). Alternate Data Streams (AltDS) support was added to NTFS for compatibility with the Macintosh's HFS file system, which used a resource stream to store icons and other file information. The use of AltDS is hidden from the user and is not accessible by conventional means. Explorer and other applications work with standard stream and cannot read data from alternatives. With AltDS, you can easily hide data that cannot be detected by standard system checks. This article will provide basic information about how AltDS works and how it is defined.

AltDS creation

It is very easy to create AltDS. For this we will use command line... First, let's create base file to which we will attach our streams.

C: \> echo Just a plan text file> sample.txt

C: \> type sample.txt
Just a plan text file

Next, we'll use a colon as an operator to indicate that we'll be using AltDS:

C: \\> echo You can "t see me> sample.txt: secret.txt

You can use the following commands to view the content:

C: \ more< sample.txt:secret.txt

or

C: \ notepad sample.txt: secret.txt

If everything works well, you will see the text: You can "t see me, and when you open it from the explorer, this text will not be visible. AltDS can also be attached not only to the file, but also to the folder. To do this, create a folder and attach some some text:

C: \> md stuff
C: \> cd stuff
C: \ stuff> echo Hide stuff in stuff>: hide.txt
C: \ stuff> dir
Volume in drive C has no label.
Volume Serial Number is 40CC-B506 Directory of C: \ stuff
09/28/2004 10:19 AM .
09/28/2004 10:19 AM…
0 File (s) 0 bytes2 Dir (s) 12,253,208,576 bytes free
C: \ stuff> notepad: hide.txt

Now you know how to view and edit the attached AltDS with notepad, as well as how to attach it to files and folders.

Hiding and launching applications

Hiding applications using AltDS is as easy as testing files. First, let's create the base file again:

Next, let's put our application into a stream, for example I used notepad.exe:

C: \ WINDOWS> type notepad.exe> ​​test.txt: note.exe

Now let's make sure that everything in our file is also text:

C: \ WINDOWS> type test.txt
Test

Now for the fun part, let's launch our hidden application:

C: \ WINDOWS> start. \ Test.txt: note.exe
C: \ WINDOWS>

Since this article is not a complete translation of the article taken, it is framed as a simple topic. Additional techniques can be found at the specified link.

UPD:

AltDS utilities (the list is taken from the article at the link above):

LADS - List Alternate Data Streams by Frank Heyne
www.heysoft.de/Frames/f_sw_la_en.htm

Streams.exe from SysInternals.

The NTFS file system has many interesting features, one of which is the availability of alternate data streams (ADS). Their essence is that each file in NTFS is a set of streams in which data is stored. By default, all data is in the main stream, but additional, alternative data streams can be added to the file if necessary.

Note. Alternate data streams in NTFS appeared a long time ago, back in Windows NT. They were created for compatibility with the HFS file system, which was then used on MacOS. HFS kept file data on a dedicated resource stream.

Files in NTFS are divided into attributes, one of which is $ DATA, or data attribute. Streams are additional properties of the $ DATA attribute. By default, there is one, the main thread $ DATA: ″ ″... As you can see, it has no name, therefore it is called unnamed... Also, if you wish, you can create additional, named streams, for example. $ DATA: ″ Stream1 ″... Each file in NTFS can have several data streams containing different, unrelated data.

All data written to the file goes to the main data stream by default. When we open the file, we see exactly the main stream, while the alternative streams are hidden from the user and are not displayed using conventional means. They cannot be seen standard ways, although some programs can read the data hidden in them. You can also use the command line to work with streams.

For example, let's open the console and use the echo command to create a text file streams.txt and write the text into it:

echo This is main stream> streams.txt

And with the following command, write the text to the alternate stream stream1:

echo This is alternate stream> streams.txt: stream1

If you now open the streams.txt file in any text editor, then we will see only the first entry, the text "This is alternate stream" will remain hidden. You can read the information hidden in stream1 with the command:

more

Alternative streams can be added not only to individual files, but also to directories. For example, let's add an alternative stream stream2 containing the text "Hide stream in Streams" to the current Streams directory:

echo Hide stream in Streams>: stream2

And we will display the stream stream2 with the following command:

more<:stream2

The content of alternate streams can be opened in more than just the console. For example, Notepad can also access data hidden in streams, if you specify the name of an alternate stream in the file name separated by a colon. Let's repeat the previous example, slightly changing the name of the stream to stream1.txt:

echo This is alternate stream> streams.txt: stream1.txt

And open an alternate stream in notepad with the command:

notepad streams.txt: stream1.txt

Note. Standard Notepad requires the txt extension in the stream name, otherwise it will not be able to open it. More advanced editors, for example the same Notepad ++, can show the content of the alternate stream regardless of its name.

The presence of alternative streams for a file is not displayed in any way in Explorer and others file managers... To find them, the easiest way is to use the command dir / R(beginning with Windows Vista), which shows all data streams, including alternative ones.

You might think that the use of alternate streams is limited to textual data. This is not at all the case, and absolutely any information can be stored in alternative streams. For example, let's create a file picture.txt and add the pic1.jpg stream to it, into which we place the image of the same name:

echo Picture> picture.txt
type pic1.jpg> picture.jpg: pic1.jpg

Thus, outwardly we have a regular text file, and to open an image from an alternate stream in graphic editor Paint we use the command:

mspaint picture.txt: pic1.jpg

Similarly, you can add any data to any type of file - add images to text files, add text information etc. Interestingly, alternative content does not increase the apparent file size, for example adding to 1kB text file 30GB HD video, file explorer will still show 1kB file size.

You can also hide executable files in alternative streams. For example, let's take the test.txt file and add the Notepad application (notepad.exe) to the alternate note.exe stream:

type notepad.exe> ​​test.txt: note.exe

And to launch a hidden notepad, use the command:

start. \ test.txt: note.exe

By the way, some malicious programs use this opportunity by adding executable code to NTFS alternate streams.

Streams utility

To work with alternative streams, there are several third party utilities, for example the console utility Streams from Sysinternals. It can detect the presence of alternate streams and delete them. The utility does not require installation, it is enough to unpack it and run it. For example, let's check the presence of streams in the Streams folder with the command:

Streams.exe -s C: \ Streams

And remove the alternate streams from the streams.txt file:

Streams.exe -d C: \ Streams \ streams.txt

PowerShell

PowerShell also knows how to work with alternate streams - create, detect, display their content, and even delete. For example, let's create a text file:

New-Item -Type file -Path C: \ Streams \ stream.txt

Let's add an entry to the main stream:

Set-Content -Path C: \ Streams \ stream.txt -Value ″ Main stream ″

And to an alternate stream named Second:

Set-Content -Path C: \ Streams \ stream.txt -Value ″ Second stream ″ -Stream Second

Then we will display the contents of the main

Get-Content -Path C: \ Streams \ stream.txt

and alternate streams:

Get-Content -Path C: \ Streams \ stream.txt -Stream Second

In order to detect the presence of alternate streams, you can use the command:

Get-Item -Path C: \ Streams \ stream.txt -Stream *

And you can remove unnecessary streams with the command:

Remove-Item -Path C: \ Streams \ streams.txt -Stream *

Usage

Alternate streams are used both by Windows itself and by some programs. For example, Internet Explorer divides the network into 4 security zones and, when downloading files, adds tags to them that contain information about the zone from which they were downloaded.

These labels are stored in the alternate stream and represent a number from 0 to 4:

Internet (3)
Local network (1)
Trusted sites (2)
Dangerous Sites (4)
Local computer (0)

To verify this, let's go to the downloads folder, take the file downloaded from the Internet and check it for alternative streams. As you can see, it contains a stream named Zone.Identifier which contains the line ZoneID = 3.

This means that the file belongs to the untrusted zone of the Internet, and you need to be careful when opening it. Some programs, such as Word, read this data when you open the file and issue a warning.

Also, the File Classification Infrastracture (FCI) infrastructure is based on the use of alternate streams. From third-party programs alternate streams use some antivirus software, in particular, Kaspersky Anti-Virus stores in them the checksum obtained as a result of the scan.

However, the use of alternative streams is not limited to this, you yourself can come up with any use for them. For example, with their help, you can hide personal information from prying eyes. Files containing alternate streams can be freely copied or moved from disk to disk, all streams will be copied along with the file.

And yet, when using alternate streams, remember that they are rigidly tied to the NTFS file system. In order to use them, the files must be located on disks with NTFS, respectively, you can work with them only from under Windows. If you move the file to any other file system, then all streams except the main one will be lost. Alternative streams are also truncated when transferring files via FTP or when sending as an email attachment.
Taken from http://windowsnotes.ru/other/alternativnye-potoki-dannyx-v-ntfs/

Yet:
ADS is a built-in feature of the NTFS file system that cannot be turned off in any way.

ADS allows you to add any files to other files and even directories (!). The OS itself uses this from time to time, adding the "Zone.Identifier" stream to the files downloaded from the Internet

Zone.Identifier, by the way, can be edited in order to get rid of the warnings “this file was downloaded from the Internet. Open in Safe Mode? "

You can add a stream to any file like this:
type file1> file2: file3

try to find
dir / r

run exe like this:
start file2: file3

if it didn't work, then like this:
mklink file4 file2: file3
start file4

This, for example, will bind the calculator to the root drive C (!) And launch it through the link

Alternate Data Streams (AltDS) support was added to NTFS for compatibility with the Macintosh's HFS file system, which used a resource stream to store icons and other file information. The use of AltDS is hidden from the user and is not accessible by conventional means. Explorer and other applications work with standard stream and cannot read data from alternatives. With AltDS, you can easily hide data that cannot be detected by standard system checks. This article will provide basic information about how AltDS works and how it is defined.

AltDS creation

It is very easy to create AltDS. To do this, we will use the command line. First, let's create a base file to which we will attach our streams.

C: \> echo Just a plan text file> sample.txt

C: \> type sample.txt
Just a plan text file

Next, we'll use a colon as an operator to indicate that we'll be using AltDS:

C: \\> echo You can "t see me> sample.txt: secret.txt

You can use the following commands to view the content:

C: \ more< sample.txt:secret.txt

or

C: \ notepad sample.txt: secret.txt

If everything works well, you will see the text: You can "t see me, and when you open it from the explorer, this text will not be visible. AltDS can also be attached not only to the file, but also to the folder. To do this, create a folder and attach some some text:

C: \> md stuff
C: \> cd stuff
C: \ stuff> echo Hide stuff in stuff>: hide.txt
C: \ stuff> dir
Volume in drive C has no label.
Volume Serial Number is 40CC-B506 Directory of C: \ stuff
09/28/2004 10:19 AM .
09/28/2004 10:19 AM…
0 File (s) 0 bytes2 Dir (s) 12,253,208,576 bytes free
C: \ stuff> notepad: hide.txt

Now you know how to view and edit the attached AltDS with notepad, as well as how to attach it to files and folders.

Hiding and launching applications

Hiding applications using AltDS is as easy as testing files. First, let's create the base file again:

Next, let's put our application into a stream, for example I used notepad.exe:

C: \ WINDOWS> type notepad.exe> ​​test.txt: note.exe

Now let's make sure that everything in our file is also text:

C: \ WINDOWS> type test.txt
Test

Now for the fun part, let's launch our hidden application:

C: \ WINDOWS> start. \ Test.txt: note.exe
C: \ WINDOWS>

Since this article is not a complete translation of the article taken, it is framed as a simple topic. Additional techniques can be found at the specified link.

UPD:

AltDS utilities (the list is taken from the article at the link above):

LADS - List Alternate Data Streams by Frank Heyne
www.heysoft.de/Frames/f_sw_la_en.htm

Streams.exe from SysInternals.