Sergey Rubenovich Vartanov is a specialist in parallel computing, programming languages, control systems, image processing and pattern recognition. Ph.D. in Physics and Mathematics (1988). Creator of the CAPER Programming Language for Parallel Computing.
What are the main solutions for concurrent computations now?
The main computing parallelization solutions on the market can be divided into multiprocessor (multi-core, matrix GPUs; i.e., with high electronic integration), and multi-machine (clusters where the architectural “trendsetters” are Cray and IBM), combining homogeneous and/or heterogeneous computing devices employing communications external to computers. For example, one of the options created by Cray combined many of its vector-pipeline machines based on J-90, AMD Opteron multiprocessor for scalar computing and clusters of matrix processors from Nvidia, thereby providing the most common types of computing.
Parallel computing software, which has been developing since the beginning of the 90s, mainly relies on an extension of the well-known C / C ++, Java, and other programming languages. There are solutions based on the parallel deployment property of recursive operators, parallel interpretations of the LISP language and LISP-like languages.
The CUDA, F #, and some other languages are focused on the matrix transformations’ description and preparation on the GPU. Specialized software libraries are used that allow concurrent calculations in a cluster environment (in particular, based on MPI), OpenCL for GPUs’ programming and similar tasks. Finally, the most common parallelization tool for multiprocessor/multicore computers is the multitasking means of operating systems related to the so-called “heavyweight” concurrent processes.
What is your approach to concurrent computations?
The fact is that most programming tasks when using parallelism are aimed at increasing productivity in computational modelling problems and in processing large amounts of data. I initially aimed at tasks that simulate the behaviour of many objects.
In 1992, I set the task of creating a programming language that is fully focused on the possibility of an expressive description of concurrently executable programs for all classes of Flynn taxonomy. I have solved the programming tools development problem for many homogeneous or heterogeneous concurrent processes that perform transformations on general or shared data and process asynchronously occurring events. The following goals were set:
• Providing different interpretations of identically written language instructions for different architectures (therefore, language support is divided into syntactic and semantic features and a complex of virtual machines that are connected when necessary depending on the computing environment current properties).
• The ability to create your virtual machine that provides your own lightweight pseudo-parallel processes initiation and control, without depending on the operating systems capabilities, specific architectures, and similar things.
• Logical units of software parallelization must be defined explicitly in the programs’ structure; i.e., program texts’ structuring should meet the parallelization tasks (the language has the means of grouping variables (data) used jointly and separately by concurrent processes).
• Providing opportunities for programs’ modular organization, overlays organization, a dynamic compilation of source code modules into object modules (jitter), source and object code modules transportation to various computing facilities in a multi-machine environment.
The first two language versions were made for MS-DOS. And if DOS itself allowed starting two pseudo-parallel processes, then I had the opportunity to start several thousand processes on the 386th processor with an acceptable runtime. In 1995, the first commercial application appeared that was engaged in information analysis and rubrication. In the late 90s, the language third version was created for Windows, later – for UNIX, and other platforms. The CAPER name reflects some of the language essence: the dynamic compilation ability and parallel asynchronous events’ processing.
What was achieved on your developments’ basis and where they were applied?
CAPER is not inferior in speed to SystemC (a language for designing and verifying system-level models implemented in the C ++ library form) used in chip design, including the work simulation of many interacting structural elements (processors, memory, etc.). Moreover, unlike SystemС, Caper is faster in parallel mode, and the parallel processes number is limited by the RAM size only. Caper is 25-30% faster than Java for sequential computing and qualitatively faster when working with many parallel processes (Java developers did not understand lightweight processes, although they planned to do this in the first half of the 90s).
Today, the 5th version is developed, which has a virtual machine distributed across the processor cores as one of the features. This allowed a significant increase in computing performance. For example, you can simultaneously run over a million parallel processes that simulate cellular automation without the use of multitasking operating systems.
The asset of using the language is the solution to such problems as:
• Isolation and codification of about one and a half thousand iris elements for the Synapse Communications Inc. project (Japan).
• Image matching and comparison for search and “necessary things” recognition on images projects for NurVentures, Inc. (USA).
• Creating a fast-parallel image clustering algorithm for Memcosoft, Inc. (USA-Armenia), including a graphic editor in Caper language.
• Parallel processing of medical roentgenology data for Course-AS (Russia).
• “Nearsighted” systems simulation (cellular automaton, neural networks).
Consecutive languages’ extensions to parallel ones have little or no future. This is because they do not consider the substantial requirements of the parallelism philosophy, the structural expressive power of the programs source codes oriented to concurrent processing.
Concurrent computing is a gigantic topic, with very capacious problems in each direction. The theme for several generations of mathematicians, developers, and programmers. The separate world. If you consider that I am a sceptic (with justification!) regarding the possibilities of full-fledged quantum computers creation, then, I believe, specialists will spend the next decades in this world, and in no other.
How did your concurrent computing research begin?
It began almost immediately after graduation, when I led a group to create a management system for distributing computing complexes for space information processing. In 1982 the theory of parallel permutation systems was created in Novosibirsk. The theory is an extension of von Neumann cellular automaton. The first love. As you know, it is for a whole life.
Then I have been occupied with the issues of software automatic parallelization. And then I became convinced quite quickly that automatic parallelization is ineffective in the general case. Starting with the simplest classes of programs in which the decision statements are present (not their special cases), it is algorithmically impossible to predict the possible maximum software parallelization. Moreover, there are no universally unambiguously effective strategies for automatic calculations distribution among computers (for example, in cases where the simultaneously executable instructions, procedures, the program threads number exceeds the physical devices number). A typical example of failures in this direction is the well-known HPF project.
In general, the concurrent calculations’ algorithms and programming require not only considerable knowledge of the parallelism problem but also considerable sophistication and ingenuity. This is to answer the question of whether algorithms and programming are art or engineering. Hence the conclusion, which became an incentive for me for all subsequent work: imperative programming, i.e. programming with a direct indication of the instructions and procedures of programs that must be executed in parallel.
Interview: Ivan Stepanyan
